2023-08-07 08:53:09,686 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-08-07 08:53:09,933 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388 2023-08-07 08:53:09,948 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-08-07 08:53:09,949 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/cluster_383c75a2-fb83-c721-ca60-8c2ce8eb403c, deleteOnExit=true 2023-08-07 08:53:09,949 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-08-07 08:53:09,949 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/test.cache.data in system properties and HBase conf 2023-08-07 08:53:09,950 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/hadoop.tmp.dir in system properties and HBase conf 2023-08-07 08:53:09,950 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/hadoop.log.dir in system properties and HBase conf 2023-08-07 08:53:09,951 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/mapreduce.cluster.local.dir in system properties and HBase conf 2023-08-07 08:53:09,951 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-08-07 08:53:09,951 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-08-07 08:53:10,070 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-08-07 08:53:10,524 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-08-07 08:53:10,531 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-08-07 08:53:10,531 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-08-07 08:53:10,531 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-08-07 08:53:10,532 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-07 08:53:10,532 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-08-07 08:53:10,533 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-08-07 08:53:10,533 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-07 08:53:10,534 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-07 08:53:10,534 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-08-07 08:53:10,535 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/nfs.dump.dir in system properties and HBase conf 2023-08-07 08:53:10,535 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/java.io.tmpdir in system properties and HBase conf 2023-08-07 08:53:10,535 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-07 08:53:10,536 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-08-07 08:53:10,536 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-08-07 08:53:11,199 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-07 08:53:11,203 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-07 08:53:11,497 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-08-07 08:53:11,730 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-08-07 08:53:11,753 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-07 08:53:11,802 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-08-07 08:53:11,849 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/java.io.tmpdir/Jetty_localhost_40021_hdfs____85uhi5/webapp 2023-08-07 08:53:12,031 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40021 2023-08-07 08:53:12,051 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-07 08:53:12,051 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-07 08:53:12,642 WARN [Listener at localhost/45787] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-07 08:53:12,720 WARN [Listener at localhost/45787] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-07 08:53:12,745 WARN [Listener at localhost/45787] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-07 08:53:12,753 INFO [Listener at localhost/45787] log.Slf4jLog(67): jetty-6.1.26 2023-08-07 08:53:12,760 INFO [Listener at localhost/45787] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/java.io.tmpdir/Jetty_localhost_37887_datanode____u9a7qy/webapp 2023-08-07 08:53:12,900 INFO [Listener at localhost/45787] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37887 2023-08-07 08:53:13,291 WARN [Listener at localhost/36443] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-07 08:53:13,316 WARN [Listener at localhost/36443] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-07 08:53:13,322 WARN [Listener at localhost/36443] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-07 08:53:13,324 INFO [Listener at localhost/36443] log.Slf4jLog(67): jetty-6.1.26 2023-08-07 08:53:13,333 INFO [Listener at localhost/36443] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/java.io.tmpdir/Jetty_localhost_35171_datanode____.ivhf1s/webapp 2023-08-07 08:53:13,479 INFO [Listener at localhost/36443] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35171 2023-08-07 08:53:13,513 WARN [Listener at localhost/39259] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-07 08:53:13,548 WARN [Listener at localhost/39259] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-07 08:53:13,559 WARN [Listener at localhost/39259] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-07 08:53:13,563 INFO [Listener at localhost/39259] log.Slf4jLog(67): jetty-6.1.26 2023-08-07 08:53:13,574 INFO [Listener at localhost/39259] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/java.io.tmpdir/Jetty_localhost_33653_datanode____.cddov9/webapp 2023-08-07 08:53:13,731 INFO [Listener at localhost/39259] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33653 2023-08-07 08:53:13,765 WARN [Listener at localhost/33561] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-07 08:53:13,927 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x4a697709d38e4259: Processing first storage report for DS-3c111aed-f33e-4581-835e-4581ab97ab31 from datanode 4fd49325-1f52-494b-bfd8-92875de86a16 2023-08-07 08:53:13,929 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x4a697709d38e4259: from storage DS-3c111aed-f33e-4581-835e-4581ab97ab31 node DatanodeRegistration(127.0.0.1:37851, datanodeUuid=4fd49325-1f52-494b-bfd8-92875de86a16, infoPort=45815, infoSecurePort=0, ipcPort=36443, storageInfo=lv=-57;cid=testClusterID;nsid=1726310995;c=1691398391279), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-08-07 08:53:13,929 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xf6dab3becf1fe4: Processing first storage report for DS-db9e2e36-3118-4e5c-b4d6-48b0f65c9427 from datanode 0712b291-8bcc-4f35-bb85-15469634c482 2023-08-07 08:53:13,929 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xf6dab3becf1fe4: from storage DS-db9e2e36-3118-4e5c-b4d6-48b0f65c9427 node DatanodeRegistration(127.0.0.1:37729, datanodeUuid=0712b291-8bcc-4f35-bb85-15469634c482, infoPort=38027, infoSecurePort=0, ipcPort=39259, storageInfo=lv=-57;cid=testClusterID;nsid=1726310995;c=1691398391279), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-07 08:53:13,929 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x4a697709d38e4259: Processing first storage report for DS-9deb361d-d65f-4377-a48a-c54abfd50fc4 from datanode 4fd49325-1f52-494b-bfd8-92875de86a16 2023-08-07 08:53:13,929 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x4a697709d38e4259: from storage DS-9deb361d-d65f-4377-a48a-c54abfd50fc4 node DatanodeRegistration(127.0.0.1:37851, datanodeUuid=4fd49325-1f52-494b-bfd8-92875de86a16, infoPort=45815, infoSecurePort=0, ipcPort=36443, storageInfo=lv=-57;cid=testClusterID;nsid=1726310995;c=1691398391279), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-07 08:53:13,930 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xf6dab3becf1fe4: Processing first storage report for DS-fea7f858-3867-4934-aa8e-219d79264811 from datanode 0712b291-8bcc-4f35-bb85-15469634c482 2023-08-07 08:53:13,930 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xf6dab3becf1fe4: from storage DS-fea7f858-3867-4934-aa8e-219d79264811 node DatanodeRegistration(127.0.0.1:37729, datanodeUuid=0712b291-8bcc-4f35-bb85-15469634c482, infoPort=38027, infoSecurePort=0, ipcPort=39259, storageInfo=lv=-57;cid=testClusterID;nsid=1726310995;c=1691398391279), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-07 08:53:13,934 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x288ab447dc0999f1: Processing first storage report for DS-aae3f3af-a59f-49ff-856b-387a4235e3f8 from datanode 5daa8478-5963-4c33-b92f-e21f18db34d9 2023-08-07 08:53:13,934 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x288ab447dc0999f1: from storage DS-aae3f3af-a59f-49ff-856b-387a4235e3f8 node DatanodeRegistration(127.0.0.1:41113, datanodeUuid=5daa8478-5963-4c33-b92f-e21f18db34d9, infoPort=34025, infoSecurePort=0, ipcPort=33561, storageInfo=lv=-57;cid=testClusterID;nsid=1726310995;c=1691398391279), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-07 08:53:13,935 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x288ab447dc0999f1: Processing first storage report for DS-b6268e68-7ce1-4f94-b4fd-a036f0d7e335 from datanode 5daa8478-5963-4c33-b92f-e21f18db34d9 2023-08-07 08:53:13,935 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x288ab447dc0999f1: from storage DS-b6268e68-7ce1-4f94-b4fd-a036f0d7e335 node DatanodeRegistration(127.0.0.1:41113, datanodeUuid=5daa8478-5963-4c33-b92f-e21f18db34d9, infoPort=34025, infoSecurePort=0, ipcPort=33561, storageInfo=lv=-57;cid=testClusterID;nsid=1726310995;c=1691398391279), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-07 08:53:14,199 DEBUG [Listener at localhost/33561] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388 2023-08-07 08:53:14,270 INFO [Listener at localhost/33561] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/cluster_383c75a2-fb83-c721-ca60-8c2ce8eb403c/zookeeper_0, clientPort=57521, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/cluster_383c75a2-fb83-c721-ca60-8c2ce8eb403c/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/cluster_383c75a2-fb83-c721-ca60-8c2ce8eb403c/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-08-07 08:53:14,284 INFO [Listener at localhost/33561] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=57521 2023-08-07 08:53:14,293 INFO [Listener at localhost/33561] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 08:53:14,296 INFO [Listener at localhost/33561] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 08:53:15,010 INFO [Listener at localhost/33561] util.FSUtils(471): Created version file at hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04 with version=8 2023-08-07 08:53:15,011 INFO [Listener at localhost/33561] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/hbase-staging 2023-08-07 08:53:15,346 INFO [Listener at localhost/33561] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-08-07 08:53:15,840 INFO [Listener at localhost/33561] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-08-07 08:53:15,872 INFO [Listener at localhost/33561] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 08:53:15,873 INFO [Listener at localhost/33561] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-07 08:53:15,873 INFO [Listener at localhost/33561] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-07 08:53:15,873 INFO [Listener at localhost/33561] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 08:53:15,873 INFO [Listener at localhost/33561] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-07 08:53:16,015 INFO [Listener at localhost/33561] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-08-07 08:53:16,087 DEBUG [Listener at localhost/33561] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-08-07 08:53:16,179 INFO [Listener at localhost/33561] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33357 2023-08-07 08:53:16,189 INFO [Listener at localhost/33561] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 08:53:16,191 INFO [Listener at localhost/33561] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 08:53:16,212 INFO [Listener at localhost/33561] zookeeper.RecoverableZooKeeper(93): Process identifier=master:33357 connecting to ZooKeeper ensemble=127.0.0.1:57521 2023-08-07 08:53:16,256 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:333570x0, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 08:53:16,260 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:33357-0x101dea1d48f0000 connected 2023-08-07 08:53:16,287 DEBUG [Listener at localhost/33561] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-07 08:53:16,288 DEBUG [Listener at localhost/33561] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-07 08:53:16,292 DEBUG [Listener at localhost/33561] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-07 08:53:16,300 DEBUG [Listener at localhost/33561] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33357 2023-08-07 08:53:16,300 DEBUG [Listener at localhost/33561] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33357 2023-08-07 08:53:16,301 DEBUG [Listener at localhost/33561] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33357 2023-08-07 08:53:16,301 DEBUG [Listener at localhost/33561] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33357 2023-08-07 08:53:16,302 DEBUG [Listener at localhost/33561] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33357 2023-08-07 08:53:16,308 INFO [Listener at localhost/33561] master.HMaster(444): hbase.rootdir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04, hbase.cluster.distributed=false 2023-08-07 08:53:16,378 INFO [Listener at localhost/33561] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-08-07 08:53:16,378 INFO [Listener at localhost/33561] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 08:53:16,379 INFO [Listener at localhost/33561] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-07 08:53:16,379 INFO [Listener at localhost/33561] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-07 08:53:16,379 INFO [Listener at localhost/33561] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 08:53:16,379 INFO [Listener at localhost/33561] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-07 08:53:16,384 INFO [Listener at localhost/33561] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-07 08:53:16,387 INFO [Listener at localhost/33561] ipc.NettyRpcServer(120): Bind to /172.31.14.131:46525 2023-08-07 08:53:16,390 INFO [Listener at localhost/33561] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-07 08:53:16,397 DEBUG [Listener at localhost/33561] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-07 08:53:16,398 INFO [Listener at localhost/33561] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 08:53:16,400 INFO [Listener at localhost/33561] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 08:53:16,402 INFO [Listener at localhost/33561] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:46525 connecting to ZooKeeper ensemble=127.0.0.1:57521 2023-08-07 08:53:16,406 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:465250x0, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 08:53:16,407 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:46525-0x101dea1d48f0001 connected 2023-08-07 08:53:16,407 DEBUG [Listener at localhost/33561] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-07 08:53:16,408 DEBUG [Listener at localhost/33561] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-07 08:53:16,409 DEBUG [Listener at localhost/33561] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-07 08:53:16,410 DEBUG [Listener at localhost/33561] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46525 2023-08-07 08:53:16,410 DEBUG [Listener at localhost/33561] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46525 2023-08-07 08:53:16,411 DEBUG [Listener at localhost/33561] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46525 2023-08-07 08:53:16,414 DEBUG [Listener at localhost/33561] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46525 2023-08-07 08:53:16,414 DEBUG [Listener at localhost/33561] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46525 2023-08-07 08:53:16,428 INFO [Listener at localhost/33561] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-08-07 08:53:16,428 INFO [Listener at localhost/33561] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 08:53:16,429 INFO [Listener at localhost/33561] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-07 08:53:16,429 INFO [Listener at localhost/33561] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-07 08:53:16,429 INFO [Listener at localhost/33561] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 08:53:16,430 INFO [Listener at localhost/33561] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-07 08:53:16,430 INFO [Listener at localhost/33561] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-07 08:53:16,434 INFO [Listener at localhost/33561] ipc.NettyRpcServer(120): Bind to /172.31.14.131:34633 2023-08-07 08:53:16,435 INFO [Listener at localhost/33561] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-07 08:53:16,436 DEBUG [Listener at localhost/33561] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-07 08:53:16,436 INFO [Listener at localhost/33561] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 08:53:16,438 INFO [Listener at localhost/33561] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 08:53:16,440 INFO [Listener at localhost/33561] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:34633 connecting to ZooKeeper ensemble=127.0.0.1:57521 2023-08-07 08:53:16,444 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:346330x0, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 08:53:16,445 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:34633-0x101dea1d48f0002 connected 2023-08-07 08:53:16,445 DEBUG [Listener at localhost/33561] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-07 08:53:16,446 DEBUG [Listener at localhost/33561] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-07 08:53:16,447 DEBUG [Listener at localhost/33561] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-07 08:53:16,449 DEBUG [Listener at localhost/33561] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34633 2023-08-07 08:53:16,450 DEBUG [Listener at localhost/33561] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34633 2023-08-07 08:53:16,451 DEBUG [Listener at localhost/33561] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34633 2023-08-07 08:53:16,455 DEBUG [Listener at localhost/33561] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34633 2023-08-07 08:53:16,455 DEBUG [Listener at localhost/33561] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34633 2023-08-07 08:53:16,467 INFO [Listener at localhost/33561] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-08-07 08:53:16,467 INFO [Listener at localhost/33561] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 08:53:16,468 INFO [Listener at localhost/33561] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-07 08:53:16,468 INFO [Listener at localhost/33561] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-07 08:53:16,468 INFO [Listener at localhost/33561] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 08:53:16,468 INFO [Listener at localhost/33561] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-07 08:53:16,468 INFO [Listener at localhost/33561] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-07 08:53:16,469 INFO [Listener at localhost/33561] ipc.NettyRpcServer(120): Bind to /172.31.14.131:34559 2023-08-07 08:53:16,470 INFO [Listener at localhost/33561] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-07 08:53:16,474 DEBUG [Listener at localhost/33561] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-07 08:53:16,475 INFO [Listener at localhost/33561] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 08:53:16,478 INFO [Listener at localhost/33561] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 08:53:16,479 INFO [Listener at localhost/33561] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:34559 connecting to ZooKeeper ensemble=127.0.0.1:57521 2023-08-07 08:53:16,483 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:345590x0, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 08:53:16,484 DEBUG [Listener at localhost/33561] zookeeper.ZKUtil(164): regionserver:345590x0, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-07 08:53:16,485 DEBUG [Listener at localhost/33561] zookeeper.ZKUtil(164): regionserver:345590x0, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-07 08:53:16,486 DEBUG [Listener at localhost/33561] zookeeper.ZKUtil(164): regionserver:345590x0, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-07 08:53:16,486 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:34559-0x101dea1d48f0003 connected 2023-08-07 08:53:16,487 DEBUG [Listener at localhost/33561] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34559 2023-08-07 08:53:16,487 DEBUG [Listener at localhost/33561] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34559 2023-08-07 08:53:16,490 DEBUG [Listener at localhost/33561] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34559 2023-08-07 08:53:16,491 DEBUG [Listener at localhost/33561] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34559 2023-08-07 08:53:16,493 DEBUG [Listener at localhost/33561] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34559 2023-08-07 08:53:16,495 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,33357,1691398395153 2023-08-07 08:53:16,504 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-07 08:53:16,505 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,33357,1691398395153 2023-08-07 08:53:16,524 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-07 08:53:16,524 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-07 08:53:16,524 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 08:53:16,524 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-07 08:53:16,524 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-07 08:53:16,526 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-07 08:53:16,528 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,33357,1691398395153 from backup master directory 2023-08-07 08:53:16,528 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-07 08:53:16,532 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,33357,1691398395153 2023-08-07 08:53:16,532 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-07 08:53:16,533 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-07 08:53:16,533 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,33357,1691398395153 2023-08-07 08:53:16,536 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-08-07 08:53:16,537 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-08-07 08:53:16,628 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/hbase.id with ID: 189a1fb4-9397-4cf5-bc03-3e41ed01da35 2023-08-07 08:53:16,672 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 08:53:16,687 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 08:53:16,779 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x38c38623 to 127.0.0.1:57521 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-07 08:53:16,811 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@653c8095, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 08:53:16,834 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-07 08:53:16,836 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-08-07 08:53:16,855 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-08-07 08:53:16,855 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-08-07 08:53:16,857 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-08-07 08:53:16,860 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-08-07 08:53:16,861 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 08:53:16,895 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/MasterData/data/master/store-tmp 2023-08-07 08:53:16,930 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:53:16,930 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-07 08:53:16,930 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 08:53:16,930 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 08:53:16,930 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-07 08:53:16,930 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 08:53:16,930 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 08:53:16,930 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-07 08:53:16,932 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/MasterData/WALs/jenkins-hbase4.apache.org,33357,1691398395153 2023-08-07 08:53:16,960 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33357%2C1691398395153, suffix=, logDir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/MasterData/WALs/jenkins-hbase4.apache.org,33357,1691398395153, archiveDir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/MasterData/oldWALs, maxLogs=10 2023-08-07 08:53:17,018 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37729,DS-db9e2e36-3118-4e5c-b4d6-48b0f65c9427,DISK] 2023-08-07 08:53:17,018 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41113,DS-aae3f3af-a59f-49ff-856b-387a4235e3f8,DISK] 2023-08-07 08:53:17,018 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37851,DS-3c111aed-f33e-4581-835e-4581ab97ab31,DISK] 2023-08-07 08:53:17,026 DEBUG [RS-EventLoopGroup-5-1] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-08-07 08:53:17,090 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/MasterData/WALs/jenkins-hbase4.apache.org,33357,1691398395153/jenkins-hbase4.apache.org%2C33357%2C1691398395153.1691398396972 2023-08-07 08:53:17,091 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37851,DS-3c111aed-f33e-4581-835e-4581ab97ab31,DISK], DatanodeInfoWithStorage[127.0.0.1:41113,DS-aae3f3af-a59f-49ff-856b-387a4235e3f8,DISK], DatanodeInfoWithStorage[127.0.0.1:37729,DS-db9e2e36-3118-4e5c-b4d6-48b0f65c9427,DISK]] 2023-08-07 08:53:17,091 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-08-07 08:53:17,092 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:53:17,095 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-08-07 08:53:17,096 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-08-07 08:53:17,151 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-08-07 08:53:17,159 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-08-07 08:53:17,182 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-08-07 08:53:17,194 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:17,199 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-07 08:53:17,201 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-07 08:53:17,215 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-08-07 08:53:17,219 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 08:53:17,220 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70453249, jitterRate=0.049835219979286194}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-07 08:53:17,220 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-07 08:53:17,222 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-08-07 08:53:17,241 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:53:17,241 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:53:17,243 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:53:17,245 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-08-07 08:53:17,277 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 32 msec 2023-08-07 08:53:17,277 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-08-07 08:53:17,302 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-08-07 08:53:17,308 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-08-07 08:53:17,331 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-08-07 08:53:17,334 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-08-07 08:53:17,336 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-08-07 08:53:17,341 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-08-07 08:53:17,345 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-08-07 08:53:17,347 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 08:53:17,348 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-08-07 08:53:17,349 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-08-07 08:53:17,361 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-08-07 08:53:17,371 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-07 08:53:17,371 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-07 08:53:17,371 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-07 08:53:17,372 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 08:53:17,371 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-07 08:53:17,372 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,33357,1691398395153, sessionid=0x101dea1d48f0000, setting cluster-up flag (Was=false) 2023-08-07 08:53:17,387 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 08:53:17,392 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-08-07 08:53:17,394 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,33357,1691398395153 2023-08-07 08:53:17,399 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 08:53:17,404 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-08-07 08:53:17,406 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,33357,1691398395153 2023-08-07 08:53:17,408 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.hbase-snapshot/.tmp 2023-08-07 08:53:17,497 INFO [RS:1;jenkins-hbase4:34633] regionserver.HRegionServer(951): ClusterId : 189a1fb4-9397-4cf5-bc03-3e41ed01da35 2023-08-07 08:53:17,499 INFO [RS:2;jenkins-hbase4:34559] regionserver.HRegionServer(951): ClusterId : 189a1fb4-9397-4cf5-bc03-3e41ed01da35 2023-08-07 08:53:17,497 INFO [RS:0;jenkins-hbase4:46525] regionserver.HRegionServer(951): ClusterId : 189a1fb4-9397-4cf5-bc03-3e41ed01da35 2023-08-07 08:53:17,504 DEBUG [RS:2;jenkins-hbase4:34559] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-07 08:53:17,504 DEBUG [RS:0;jenkins-hbase4:46525] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-07 08:53:17,504 DEBUG [RS:1;jenkins-hbase4:34633] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-07 08:53:17,512 DEBUG [RS:2;jenkins-hbase4:34559] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-07 08:53:17,512 DEBUG [RS:0;jenkins-hbase4:46525] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-07 08:53:17,512 DEBUG [RS:1;jenkins-hbase4:34633] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-07 08:53:17,512 DEBUG [RS:0;jenkins-hbase4:46525] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-07 08:53:17,512 DEBUG [RS:2;jenkins-hbase4:34559] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-07 08:53:17,512 DEBUG [RS:1;jenkins-hbase4:34633] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-07 08:53:17,516 DEBUG [RS:0;jenkins-hbase4:46525] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-07 08:53:17,517 DEBUG [RS:2;jenkins-hbase4:34559] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-07 08:53:17,517 DEBUG [RS:1;jenkins-hbase4:34633] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-07 08:53:17,518 DEBUG [RS:0;jenkins-hbase4:46525] zookeeper.ReadOnlyZKClient(139): Connect 0x26051c8e to 127.0.0.1:57521 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-07 08:53:17,519 DEBUG [RS:1;jenkins-hbase4:34633] zookeeper.ReadOnlyZKClient(139): Connect 0x68f1cf0c to 127.0.0.1:57521 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-07 08:53:17,519 DEBUG [RS:2;jenkins-hbase4:34559] zookeeper.ReadOnlyZKClient(139): Connect 0x7649fe0f to 127.0.0.1:57521 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-07 08:53:17,531 DEBUG [RS:0;jenkins-hbase4:46525] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@28c2e63e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 08:53:17,531 DEBUG [RS:1;jenkins-hbase4:34633] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@66362c62, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 08:53:17,532 DEBUG [RS:0;jenkins-hbase4:46525] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2cfd77fe, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-07 08:53:17,532 DEBUG [RS:1;jenkins-hbase4:34633] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6209a47c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-07 08:53:17,533 DEBUG [RS:2;jenkins-hbase4:34559] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@788e0eb6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 08:53:17,533 DEBUG [RS:2;jenkins-hbase4:34559] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7cb1957f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-07 08:53:17,535 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-08-07 08:53:17,545 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 08:53:17,546 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 08:53:17,546 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 08:53:17,546 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 08:53:17,546 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-08-07 08:53:17,546 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 08:53:17,546 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-07 08:53:17,546 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 08:53:17,547 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398427547 2023-08-07 08:53:17,549 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-08-07 08:53:17,556 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-08-07 08:53:17,557 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-08-07 08:53:17,562 DEBUG [RS:1;jenkins-hbase4:34633] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:34633 2023-08-07 08:53:17,562 DEBUG [RS:2;jenkins-hbase4:34559] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:34559 2023-08-07 08:53:17,563 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-07 08:53:17,566 INFO [RS:2;jenkins-hbase4:34559] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-07 08:53:17,566 INFO [RS:1;jenkins-hbase4:34633] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-07 08:53:17,567 INFO [RS:1;jenkins-hbase4:34633] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-07 08:53:17,564 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-08-07 08:53:17,563 DEBUG [RS:0;jenkins-hbase4:46525] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:46525 2023-08-07 08:53:17,567 DEBUG [RS:1;jenkins-hbase4:34633] regionserver.HRegionServer(1022): About to register with Master. 2023-08-07 08:53:17,567 INFO [RS:2;jenkins-hbase4:34559] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-07 08:53:17,568 INFO [RS:0;jenkins-hbase4:46525] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-07 08:53:17,568 INFO [RS:0;jenkins-hbase4:46525] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-07 08:53:17,568 DEBUG [RS:2;jenkins-hbase4:34559] regionserver.HRegionServer(1022): About to register with Master. 2023-08-07 08:53:17,568 DEBUG [RS:0;jenkins-hbase4:46525] regionserver.HRegionServer(1022): About to register with Master. 2023-08-07 08:53:17,571 INFO [RS:2;jenkins-hbase4:34559] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,33357,1691398395153 with isa=jenkins-hbase4.apache.org/172.31.14.131:34559, startcode=1691398396467 2023-08-07 08:53:17,571 INFO [RS:0;jenkins-hbase4:46525] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,33357,1691398395153 with isa=jenkins-hbase4.apache.org/172.31.14.131:46525, startcode=1691398396377 2023-08-07 08:53:17,571 INFO [RS:1;jenkins-hbase4:34633] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,33357,1691398395153 with isa=jenkins-hbase4.apache.org/172.31.14.131:34633, startcode=1691398396427 2023-08-07 08:53:17,577 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-08-07 08:53:17,577 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-08-07 08:53:17,578 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-08-07 08:53:17,578 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-08-07 08:53:17,578 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-07 08:53:17,581 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-08-07 08:53:17,583 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-08-07 08:53:17,583 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-08-07 08:53:17,587 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-08-07 08:53:17,588 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-08-07 08:53:17,590 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691398397590,5,FailOnTimeoutGroup] 2023-08-07 08:53:17,590 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691398397590,5,FailOnTimeoutGroup] 2023-08-07 08:53:17,590 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-07 08:53:17,590 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-08-07 08:53:17,592 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-08-07 08:53:17,592 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-08-07 08:53:17,595 DEBUG [RS:1;jenkins-hbase4:34633] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-07 08:53:17,595 DEBUG [RS:0;jenkins-hbase4:46525] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-07 08:53:17,596 DEBUG [RS:2;jenkins-hbase4:34559] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-07 08:53:17,624 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-07 08:53:17,625 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-07 08:53:17,625 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04 2023-08-07 08:53:17,660 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:53:17,663 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-07 08:53:17,666 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/meta/1588230740/info 2023-08-07 08:53:17,667 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-07 08:53:17,668 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:17,669 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-07 08:53:17,672 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/meta/1588230740/rep_barrier 2023-08-07 08:53:17,672 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40821, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-08-07 08:53:17,672 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55139, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-08-07 08:53:17,672 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46909, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-08-07 08:53:17,674 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-07 08:53:17,675 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:17,675 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-07 08:53:17,677 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/meta/1588230740/table 2023-08-07 08:53:17,678 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-07 08:53:17,679 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:17,681 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/meta/1588230740 2023-08-07 08:53:17,682 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/meta/1588230740 2023-08-07 08:53:17,686 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=33357] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:17,687 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-08-07 08:53:17,689 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-07 08:53:17,693 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 08:53:17,694 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64152487, jitterRate=-0.04405345022678375}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-08-07 08:53:17,694 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-07 08:53:17,694 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-07 08:53:17,694 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-07 08:53:17,694 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-07 08:53:17,694 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-07 08:53:17,694 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-07 08:53:17,695 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-07 08:53:17,695 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-07 08:53:17,698 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=33357] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:17,699 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=33357] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:17,707 DEBUG [RS:2;jenkins-hbase4:34559] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04 2023-08-07 08:53:17,708 DEBUG [RS:2;jenkins-hbase4:34559] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:45787 2023-08-07 08:53:17,708 DEBUG [RS:2;jenkins-hbase4:34559] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-07 08:53:17,707 DEBUG [RS:1;jenkins-hbase4:34633] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04 2023-08-07 08:53:17,708 DEBUG [RS:1;jenkins-hbase4:34633] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:45787 2023-08-07 08:53:17,708 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-08-07 08:53:17,708 DEBUG [RS:1;jenkins-hbase4:34633] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-07 08:53:17,709 DEBUG [RS:0;jenkins-hbase4:46525] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04 2023-08-07 08:53:17,709 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-08-07 08:53:17,709 DEBUG [RS:0;jenkins-hbase4:46525] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:45787 2023-08-07 08:53:17,709 DEBUG [RS:0;jenkins-hbase4:46525] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-07 08:53:17,718 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-07 08:53:17,719 DEBUG [RS:0;jenkins-hbase4:46525] zookeeper.ZKUtil(162): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:17,719 WARN [RS:0;jenkins-hbase4:46525] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-07 08:53:17,719 DEBUG [RS:1;jenkins-hbase4:34633] zookeeper.ZKUtil(162): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:17,719 INFO [RS:0;jenkins-hbase4:46525] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 08:53:17,719 WARN [RS:1;jenkins-hbase4:34633] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-07 08:53:17,723 DEBUG [RS:2;jenkins-hbase4:34559] zookeeper.ZKUtil(162): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:17,724 WARN [RS:2;jenkins-hbase4:34559] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-07 08:53:17,724 DEBUG [RS:0;jenkins-hbase4:46525] regionserver.HRegionServer(1948): logDir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/WALs/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:17,724 INFO [RS:1;jenkins-hbase4:34633] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 08:53:17,724 DEBUG [RS:1;jenkins-hbase4:34633] regionserver.HRegionServer(1948): logDir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/WALs/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:17,725 INFO [RS:2;jenkins-hbase4:34559] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 08:53:17,726 DEBUG [RS:2;jenkins-hbase4:34559] regionserver.HRegionServer(1948): logDir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/WALs/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:17,726 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,34633,1691398396427] 2023-08-07 08:53:17,727 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,34559,1691398396467] 2023-08-07 08:53:17,727 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,46525,1691398396377] 2023-08-07 08:53:17,727 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-08-07 08:53:17,738 DEBUG [RS:1;jenkins-hbase4:34633] zookeeper.ZKUtil(162): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:17,739 DEBUG [RS:0;jenkins-hbase4:46525] zookeeper.ZKUtil(162): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:17,739 DEBUG [RS:2;jenkins-hbase4:34559] zookeeper.ZKUtil(162): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:17,739 DEBUG [RS:1;jenkins-hbase4:34633] zookeeper.ZKUtil(162): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:17,739 DEBUG [RS:0;jenkins-hbase4:46525] zookeeper.ZKUtil(162): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:17,739 DEBUG [RS:2;jenkins-hbase4:34559] zookeeper.ZKUtil(162): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:17,740 DEBUG [RS:1;jenkins-hbase4:34633] zookeeper.ZKUtil(162): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:17,740 DEBUG [RS:2;jenkins-hbase4:34559] zookeeper.ZKUtil(162): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:17,740 DEBUG [RS:0;jenkins-hbase4:46525] zookeeper.ZKUtil(162): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:17,752 DEBUG [RS:0;jenkins-hbase4:46525] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-08-07 08:53:17,752 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-08-07 08:53:17,752 DEBUG [RS:1;jenkins-hbase4:34633] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-08-07 08:53:17,752 DEBUG [RS:2;jenkins-hbase4:34559] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-08-07 08:53:17,755 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-08-07 08:53:17,762 INFO [RS:1;jenkins-hbase4:34633] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-07 08:53:17,762 INFO [RS:0;jenkins-hbase4:46525] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-07 08:53:17,763 INFO [RS:2;jenkins-hbase4:34559] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-07 08:53:17,786 INFO [RS:1;jenkins-hbase4:34633] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-07 08:53:17,786 INFO [RS:0;jenkins-hbase4:46525] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-07 08:53:17,786 INFO [RS:2;jenkins-hbase4:34559] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-07 08:53:17,790 INFO [RS:1;jenkins-hbase4:34633] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-07 08:53:17,790 INFO [RS:0;jenkins-hbase4:46525] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-07 08:53:17,790 INFO [RS:2;jenkins-hbase4:34559] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-07 08:53:17,790 INFO [RS:0;jenkins-hbase4:46525] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 08:53:17,790 INFO [RS:1;jenkins-hbase4:34633] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 08:53:17,791 INFO [RS:2;jenkins-hbase4:34559] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 08:53:17,792 INFO [RS:2;jenkins-hbase4:34559] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-07 08:53:17,792 INFO [RS:1;jenkins-hbase4:34633] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-07 08:53:17,792 INFO [RS:0;jenkins-hbase4:46525] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-07 08:53:17,801 INFO [RS:0;jenkins-hbase4:46525] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-07 08:53:17,801 INFO [RS:1;jenkins-hbase4:34633] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-07 08:53:17,801 DEBUG [RS:0;jenkins-hbase4:46525] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 08:53:17,801 INFO [RS:2;jenkins-hbase4:34559] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-07 08:53:17,801 DEBUG [RS:0;jenkins-hbase4:46525] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 08:53:17,801 DEBUG [RS:1;jenkins-hbase4:34633] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 08:53:17,801 DEBUG [RS:0;jenkins-hbase4:46525] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 08:53:17,801 DEBUG [RS:2;jenkins-hbase4:34559] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 08:53:17,802 DEBUG [RS:0;jenkins-hbase4:46525] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 08:53:17,802 DEBUG [RS:2;jenkins-hbase4:34559] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 08:53:17,802 DEBUG [RS:0;jenkins-hbase4:46525] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 08:53:17,802 DEBUG [RS:2;jenkins-hbase4:34559] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 08:53:17,802 DEBUG [RS:0;jenkins-hbase4:46525] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-07 08:53:17,802 DEBUG [RS:2;jenkins-hbase4:34559] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 08:53:17,802 DEBUG [RS:0;jenkins-hbase4:46525] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 08:53:17,802 DEBUG [RS:2;jenkins-hbase4:34559] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 08:53:17,802 DEBUG [RS:0;jenkins-hbase4:46525] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 08:53:17,802 DEBUG [RS:2;jenkins-hbase4:34559] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-07 08:53:17,802 DEBUG [RS:0;jenkins-hbase4:46525] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 08:53:17,802 DEBUG [RS:2;jenkins-hbase4:34559] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 08:53:17,802 DEBUG [RS:0;jenkins-hbase4:46525] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 08:53:17,802 DEBUG [RS:2;jenkins-hbase4:34559] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 08:53:17,802 DEBUG [RS:1;jenkins-hbase4:34633] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 08:53:17,803 DEBUG [RS:2;jenkins-hbase4:34559] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 08:53:17,803 DEBUG [RS:2;jenkins-hbase4:34559] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 08:53:17,803 DEBUG [RS:1;jenkins-hbase4:34633] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 08:53:17,803 DEBUG [RS:1;jenkins-hbase4:34633] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 08:53:17,804 DEBUG [RS:1;jenkins-hbase4:34633] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 08:53:17,804 INFO [RS:0;jenkins-hbase4:46525] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-07 08:53:17,804 DEBUG [RS:1;jenkins-hbase4:34633] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-07 08:53:17,804 INFO [RS:0;jenkins-hbase4:46525] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-07 08:53:17,804 DEBUG [RS:1;jenkins-hbase4:34633] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 08:53:17,804 INFO [RS:0;jenkins-hbase4:46525] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-07 08:53:17,804 DEBUG [RS:1;jenkins-hbase4:34633] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 08:53:17,805 DEBUG [RS:1;jenkins-hbase4:34633] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 08:53:17,805 DEBUG [RS:1;jenkins-hbase4:34633] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 08:53:17,805 INFO [RS:2;jenkins-hbase4:34559] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-07 08:53:17,805 INFO [RS:2;jenkins-hbase4:34559] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-07 08:53:17,806 INFO [RS:2;jenkins-hbase4:34559] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-07 08:53:17,806 INFO [RS:1;jenkins-hbase4:34633] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-07 08:53:17,807 INFO [RS:1;jenkins-hbase4:34633] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-07 08:53:17,807 INFO [RS:1;jenkins-hbase4:34633] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-07 08:53:17,822 INFO [RS:0;jenkins-hbase4:46525] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-07 08:53:17,823 INFO [RS:1;jenkins-hbase4:34633] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-07 08:53:17,823 INFO [RS:2;jenkins-hbase4:34559] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-07 08:53:17,825 INFO [RS:0;jenkins-hbase4:46525] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46525,1691398396377-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 08:53:17,825 INFO [RS:1;jenkins-hbase4:34633] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34633,1691398396427-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 08:53:17,825 INFO [RS:2;jenkins-hbase4:34559] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34559,1691398396467-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 08:53:17,842 INFO [RS:0;jenkins-hbase4:46525] regionserver.Replication(203): jenkins-hbase4.apache.org,46525,1691398396377 started 2023-08-07 08:53:17,843 INFO [RS:0;jenkins-hbase4:46525] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,46525,1691398396377, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:46525, sessionid=0x101dea1d48f0001 2023-08-07 08:53:17,843 DEBUG [RS:0;jenkins-hbase4:46525] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-07 08:53:17,843 DEBUG [RS:0;jenkins-hbase4:46525] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:17,843 DEBUG [RS:0;jenkins-hbase4:46525] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46525,1691398396377' 2023-08-07 08:53:17,843 DEBUG [RS:0;jenkins-hbase4:46525] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:17,844 DEBUG [RS:0;jenkins-hbase4:46525] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:17,844 DEBUG [RS:0;jenkins-hbase4:46525] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-07 08:53:17,844 DEBUG [RS:0;jenkins-hbase4:46525] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-07 08:53:17,844 DEBUG [RS:0;jenkins-hbase4:46525] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:17,844 DEBUG [RS:0;jenkins-hbase4:46525] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46525,1691398396377' 2023-08-07 08:53:17,844 DEBUG [RS:0;jenkins-hbase4:46525] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-08-07 08:53:17,845 DEBUG [RS:0;jenkins-hbase4:46525] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-08-07 08:53:17,845 DEBUG [RS:0;jenkins-hbase4:46525] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-07 08:53:17,845 INFO [RS:0;jenkins-hbase4:46525] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-07 08:53:17,846 INFO [RS:0;jenkins-hbase4:46525] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-07 08:53:17,849 INFO [RS:1;jenkins-hbase4:34633] regionserver.Replication(203): jenkins-hbase4.apache.org,34633,1691398396427 started 2023-08-07 08:53:17,849 INFO [RS:1;jenkins-hbase4:34633] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,34633,1691398396427, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:34633, sessionid=0x101dea1d48f0002 2023-08-07 08:53:17,849 INFO [RS:2;jenkins-hbase4:34559] regionserver.Replication(203): jenkins-hbase4.apache.org,34559,1691398396467 started 2023-08-07 08:53:17,849 DEBUG [RS:1;jenkins-hbase4:34633] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-07 08:53:17,850 INFO [RS:2;jenkins-hbase4:34559] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,34559,1691398396467, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:34559, sessionid=0x101dea1d48f0003 2023-08-07 08:53:17,850 DEBUG [RS:1;jenkins-hbase4:34633] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:17,850 DEBUG [RS:2;jenkins-hbase4:34559] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-07 08:53:17,851 DEBUG [RS:2;jenkins-hbase4:34559] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:17,851 DEBUG [RS:2;jenkins-hbase4:34559] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34559,1691398396467' 2023-08-07 08:53:17,851 DEBUG [RS:2;jenkins-hbase4:34559] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:17,850 DEBUG [RS:1;jenkins-hbase4:34633] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34633,1691398396427' 2023-08-07 08:53:17,851 DEBUG [RS:1;jenkins-hbase4:34633] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:17,851 DEBUG [RS:2;jenkins-hbase4:34559] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:17,851 DEBUG [RS:1;jenkins-hbase4:34633] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:17,852 DEBUG [RS:2;jenkins-hbase4:34559] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-07 08:53:17,852 DEBUG [RS:2;jenkins-hbase4:34559] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-07 08:53:17,852 DEBUG [RS:1;jenkins-hbase4:34633] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-07 08:53:17,852 DEBUG [RS:2;jenkins-hbase4:34559] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:17,853 DEBUG [RS:2;jenkins-hbase4:34559] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34559,1691398396467' 2023-08-07 08:53:17,853 DEBUG [RS:2;jenkins-hbase4:34559] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-08-07 08:53:17,852 DEBUG [RS:1;jenkins-hbase4:34633] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-07 08:53:17,853 DEBUG [RS:1;jenkins-hbase4:34633] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:17,853 DEBUG [RS:1;jenkins-hbase4:34633] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34633,1691398396427' 2023-08-07 08:53:17,853 DEBUG [RS:1;jenkins-hbase4:34633] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-08-07 08:53:17,853 DEBUG [RS:2;jenkins-hbase4:34559] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-08-07 08:53:17,853 DEBUG [RS:1;jenkins-hbase4:34633] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-08-07 08:53:17,854 DEBUG [RS:2;jenkins-hbase4:34559] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-07 08:53:17,854 INFO [RS:2;jenkins-hbase4:34559] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-07 08:53:17,854 INFO [RS:2;jenkins-hbase4:34559] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-07 08:53:17,855 DEBUG [RS:1;jenkins-hbase4:34633] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-07 08:53:17,855 INFO [RS:1;jenkins-hbase4:34633] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-07 08:53:17,855 INFO [RS:1;jenkins-hbase4:34633] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-07 08:53:17,907 DEBUG [jenkins-hbase4:33357] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-08-07 08:53:17,911 DEBUG [jenkins-hbase4:33357] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-07 08:53:17,916 DEBUG [jenkins-hbase4:33357] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-07 08:53:17,916 DEBUG [jenkins-hbase4:33357] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-07 08:53:17,916 DEBUG [jenkins-hbase4:33357] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-07 08:53:17,916 DEBUG [jenkins-hbase4:33357] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-07 08:53:17,919 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,34559,1691398396467, state=OPENING 2023-08-07 08:53:17,926 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-08-07 08:53:17,928 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 08:53:17,929 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-07 08:53:17,932 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,34559,1691398396467}] 2023-08-07 08:53:17,956 INFO [RS:0;jenkins-hbase4:46525] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46525%2C1691398396377, suffix=, logDir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/WALs/jenkins-hbase4.apache.org,46525,1691398396377, archiveDir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/oldWALs, maxLogs=32 2023-08-07 08:53:17,958 INFO [RS:1;jenkins-hbase4:34633] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34633%2C1691398396427, suffix=, logDir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/WALs/jenkins-hbase4.apache.org,34633,1691398396427, archiveDir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/oldWALs, maxLogs=32 2023-08-07 08:53:17,958 INFO [RS:2;jenkins-hbase4:34559] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34559%2C1691398396467, suffix=, logDir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/WALs/jenkins-hbase4.apache.org,34559,1691398396467, archiveDir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/oldWALs, maxLogs=32 2023-08-07 08:53:17,984 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37729,DS-db9e2e36-3118-4e5c-b4d6-48b0f65c9427,DISK] 2023-08-07 08:53:17,988 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41113,DS-aae3f3af-a59f-49ff-856b-387a4235e3f8,DISK] 2023-08-07 08:53:17,989 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37851,DS-3c111aed-f33e-4581-835e-4581ab97ab31,DISK] 2023-08-07 08:53:17,996 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37729,DS-db9e2e36-3118-4e5c-b4d6-48b0f65c9427,DISK] 2023-08-07 08:53:17,996 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37851,DS-3c111aed-f33e-4581-835e-4581ab97ab31,DISK] 2023-08-07 08:53:17,996 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41113,DS-aae3f3af-a59f-49ff-856b-387a4235e3f8,DISK] 2023-08-07 08:53:17,999 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37729,DS-db9e2e36-3118-4e5c-b4d6-48b0f65c9427,DISK] 2023-08-07 08:53:18,000 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37851,DS-3c111aed-f33e-4581-835e-4581ab97ab31,DISK] 2023-08-07 08:53:18,000 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41113,DS-aae3f3af-a59f-49ff-856b-387a4235e3f8,DISK] 2023-08-07 08:53:18,007 INFO [RS:0;jenkins-hbase4:46525] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/WALs/jenkins-hbase4.apache.org,46525,1691398396377/jenkins-hbase4.apache.org%2C46525%2C1691398396377.1691398397960 2023-08-07 08:53:18,012 DEBUG [RS:0;jenkins-hbase4:46525] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37729,DS-db9e2e36-3118-4e5c-b4d6-48b0f65c9427,DISK], DatanodeInfoWithStorage[127.0.0.1:41113,DS-aae3f3af-a59f-49ff-856b-387a4235e3f8,DISK], DatanodeInfoWithStorage[127.0.0.1:37851,DS-3c111aed-f33e-4581-835e-4581ab97ab31,DISK]] 2023-08-07 08:53:18,018 INFO [RS:1;jenkins-hbase4:34633] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/WALs/jenkins-hbase4.apache.org,34633,1691398396427/jenkins-hbase4.apache.org%2C34633%2C1691398396427.1691398397961 2023-08-07 08:53:18,018 INFO [RS:2;jenkins-hbase4:34559] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/WALs/jenkins-hbase4.apache.org,34559,1691398396467/jenkins-hbase4.apache.org%2C34559%2C1691398396467.1691398397961 2023-08-07 08:53:18,018 DEBUG [RS:1;jenkins-hbase4:34633] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41113,DS-aae3f3af-a59f-49ff-856b-387a4235e3f8,DISK], DatanodeInfoWithStorage[127.0.0.1:37851,DS-3c111aed-f33e-4581-835e-4581ab97ab31,DISK], DatanodeInfoWithStorage[127.0.0.1:37729,DS-db9e2e36-3118-4e5c-b4d6-48b0f65c9427,DISK]] 2023-08-07 08:53:18,018 DEBUG [RS:2;jenkins-hbase4:34559] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41113,DS-aae3f3af-a59f-49ff-856b-387a4235e3f8,DISK], DatanodeInfoWithStorage[127.0.0.1:37851,DS-3c111aed-f33e-4581-835e-4581ab97ab31,DISK], DatanodeInfoWithStorage[127.0.0.1:37729,DS-db9e2e36-3118-4e5c-b4d6-48b0f65c9427,DISK]] 2023-08-07 08:53:18,115 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:18,118 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-07 08:53:18,121 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45128, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-07 08:53:18,133 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-08-07 08:53:18,133 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 08:53:18,137 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34559%2C1691398396467.meta, suffix=.meta, logDir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/WALs/jenkins-hbase4.apache.org,34559,1691398396467, archiveDir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/oldWALs, maxLogs=32 2023-08-07 08:53:18,161 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41113,DS-aae3f3af-a59f-49ff-856b-387a4235e3f8,DISK] 2023-08-07 08:53:18,161 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37851,DS-3c111aed-f33e-4581-835e-4581ab97ab31,DISK] 2023-08-07 08:53:18,163 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37729,DS-db9e2e36-3118-4e5c-b4d6-48b0f65c9427,DISK] 2023-08-07 08:53:18,169 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/WALs/jenkins-hbase4.apache.org,34559,1691398396467/jenkins-hbase4.apache.org%2C34559%2C1691398396467.meta.1691398398139.meta 2023-08-07 08:53:18,170 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41113,DS-aae3f3af-a59f-49ff-856b-387a4235e3f8,DISK], DatanodeInfoWithStorage[127.0.0.1:37851,DS-3c111aed-f33e-4581-835e-4581ab97ab31,DISK], DatanodeInfoWithStorage[127.0.0.1:37729,DS-db9e2e36-3118-4e5c-b4d6-48b0f65c9427,DISK]] 2023-08-07 08:53:18,170 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-08-07 08:53:18,176 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-07 08:53:18,177 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-08-07 08:53:18,198 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-08-07 08:53:18,200 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-08-07 08:53:18,205 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-08-07 08:53:18,205 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:53:18,205 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-08-07 08:53:18,205 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-08-07 08:53:18,209 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-07 08:53:18,211 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/meta/1588230740/info 2023-08-07 08:53:18,211 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/meta/1588230740/info 2023-08-07 08:53:18,211 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-07 08:53:18,212 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:18,212 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-07 08:53:18,214 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/meta/1588230740/rep_barrier 2023-08-07 08:53:18,214 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/meta/1588230740/rep_barrier 2023-08-07 08:53:18,214 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-07 08:53:18,215 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:18,215 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-07 08:53:18,217 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/meta/1588230740/table 2023-08-07 08:53:18,217 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/meta/1588230740/table 2023-08-07 08:53:18,217 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-07 08:53:18,218 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:18,220 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/meta/1588230740 2023-08-07 08:53:18,223 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/meta/1588230740 2023-08-07 08:53:18,227 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-08-07 08:53:18,229 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-07 08:53:18,231 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68661086, jitterRate=0.023129910230636597}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-08-07 08:53:18,231 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-07 08:53:18,241 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1691398398107 2023-08-07 08:53:18,263 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-08-07 08:53:18,264 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-08-07 08:53:18,265 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,34559,1691398396467, state=OPEN 2023-08-07 08:53:18,268 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-08-07 08:53:18,268 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-07 08:53:18,272 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-08-07 08:53:18,272 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,34559,1691398396467 in 336 msec 2023-08-07 08:53:18,277 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-08-07 08:53:18,278 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 547 msec 2023-08-07 08:53:18,283 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 830 msec 2023-08-07 08:53:18,283 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1691398398283, completionTime=-1 2023-08-07 08:53:18,283 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-08-07 08:53:18,283 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:53:18,346 DEBUG [hconnection-0x2e450bc6-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 08:53:18,350 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45144, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-07 08:53:18,370 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:53:18,370 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398458370 2023-08-07 08:53:18,370 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398518370 2023-08-07 08:53:18,370 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 86 msec 2023-08-07 08:53:18,402 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33357,1691398395153-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 08:53:18,402 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33357,1691398395153-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-07 08:53:18,402 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33357,1691398395153-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-07 08:53:18,405 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:33357, period=300000, unit=MILLISECONDS is enabled. 2023-08-07 08:53:18,405 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-08-07 08:53:18,414 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-08-07 08:53:18,423 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-08-07 08:53:18,424 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-07 08:53:18,434 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-08-07 08:53:18,436 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-08-07 08:53:18,439 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-07 08:53:18,465 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/hbase/namespace/261180939175fe040900ce2fa993a79f 2023-08-07 08:53:18,468 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/hbase/namespace/261180939175fe040900ce2fa993a79f empty. 2023-08-07 08:53:18,469 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/hbase/namespace/261180939175fe040900ce2fa993a79f 2023-08-07 08:53:18,469 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-08-07 08:53:18,507 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-08-07 08:53:18,509 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 261180939175fe040900ce2fa993a79f, NAME => 'hbase:namespace,,1691398398424.261180939175fe040900ce2fa993a79f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp 2023-08-07 08:53:18,526 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1691398398424.261180939175fe040900ce2fa993a79f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:53:18,526 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 261180939175fe040900ce2fa993a79f, disabling compactions & flushes 2023-08-07 08:53:18,526 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1691398398424.261180939175fe040900ce2fa993a79f. 2023-08-07 08:53:18,526 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1691398398424.261180939175fe040900ce2fa993a79f. 2023-08-07 08:53:18,526 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1691398398424.261180939175fe040900ce2fa993a79f. after waiting 0 ms 2023-08-07 08:53:18,526 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1691398398424.261180939175fe040900ce2fa993a79f. 2023-08-07 08:53:18,526 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1691398398424.261180939175fe040900ce2fa993a79f. 2023-08-07 08:53:18,526 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 261180939175fe040900ce2fa993a79f: 2023-08-07 08:53:18,531 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-08-07 08:53:18,545 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1691398398424.261180939175fe040900ce2fa993a79f.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691398398534"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398398534"}]},"ts":"1691398398534"} 2023-08-07 08:53:18,575 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-07 08:53:18,577 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-07 08:53:18,583 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398398577"}]},"ts":"1691398398577"} 2023-08-07 08:53:18,587 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-08-07 08:53:18,592 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-07 08:53:18,594 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-07 08:53:18,594 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-07 08:53:18,594 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-07 08:53:18,594 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-07 08:53:18,597 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=261180939175fe040900ce2fa993a79f, ASSIGN}] 2023-08-07 08:53:18,601 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=261180939175fe040900ce2fa993a79f, ASSIGN 2023-08-07 08:53:18,603 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=261180939175fe040900ce2fa993a79f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46525,1691398396377; forceNewPlan=false, retain=false 2023-08-07 08:53:18,755 INFO [jenkins-hbase4:33357] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-07 08:53:18,756 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=261180939175fe040900ce2fa993a79f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:18,757 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1691398398424.261180939175fe040900ce2fa993a79f.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691398398756"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398398756"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398398756"}]},"ts":"1691398398756"} 2023-08-07 08:53:18,764 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 261180939175fe040900ce2fa993a79f, server=jenkins-hbase4.apache.org,46525,1691398396377}] 2023-08-07 08:53:18,919 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:18,919 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-07 08:53:18,923 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54384, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-07 08:53:18,930 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1691398398424.261180939175fe040900ce2fa993a79f. 2023-08-07 08:53:18,932 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 261180939175fe040900ce2fa993a79f, NAME => 'hbase:namespace,,1691398398424.261180939175fe040900ce2fa993a79f.', STARTKEY => '', ENDKEY => ''} 2023-08-07 08:53:18,932 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-07 08:53:18,933 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 261180939175fe040900ce2fa993a79f 2023-08-07 08:53:18,933 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1691398398424.261180939175fe040900ce2fa993a79f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:53:18,933 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 261180939175fe040900ce2fa993a79f 2023-08-07 08:53:18,933 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 261180939175fe040900ce2fa993a79f 2023-08-07 08:53:18,936 INFO [StoreOpener-261180939175fe040900ce2fa993a79f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 261180939175fe040900ce2fa993a79f 2023-08-07 08:53:18,938 DEBUG [StoreOpener-261180939175fe040900ce2fa993a79f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/namespace/261180939175fe040900ce2fa993a79f/info 2023-08-07 08:53:18,938 DEBUG [StoreOpener-261180939175fe040900ce2fa993a79f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/namespace/261180939175fe040900ce2fa993a79f/info 2023-08-07 08:53:18,939 INFO [StoreOpener-261180939175fe040900ce2fa993a79f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 261180939175fe040900ce2fa993a79f columnFamilyName info 2023-08-07 08:53:18,940 INFO [StoreOpener-261180939175fe040900ce2fa993a79f-1] regionserver.HStore(310): Store=261180939175fe040900ce2fa993a79f/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:18,941 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/namespace/261180939175fe040900ce2fa993a79f 2023-08-07 08:53:18,943 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/namespace/261180939175fe040900ce2fa993a79f 2023-08-07 08:53:18,947 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 261180939175fe040900ce2fa993a79f 2023-08-07 08:53:18,950 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/namespace/261180939175fe040900ce2fa993a79f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 08:53:18,951 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 261180939175fe040900ce2fa993a79f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62006783, jitterRate=-0.07602693140506744}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-07 08:53:18,951 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 261180939175fe040900ce2fa993a79f: 2023-08-07 08:53:18,955 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1691398398424.261180939175fe040900ce2fa993a79f., pid=6, masterSystemTime=1691398398918 2023-08-07 08:53:18,960 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1691398398424.261180939175fe040900ce2fa993a79f. 2023-08-07 08:53:18,960 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1691398398424.261180939175fe040900ce2fa993a79f. 2023-08-07 08:53:18,962 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=261180939175fe040900ce2fa993a79f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:18,962 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1691398398424.261180939175fe040900ce2fa993a79f.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691398398961"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691398398961"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691398398961"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398398961"}]},"ts":"1691398398961"} 2023-08-07 08:53:18,969 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-08-07 08:53:18,970 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 261180939175fe040900ce2fa993a79f, server=jenkins-hbase4.apache.org,46525,1691398396377 in 201 msec 2023-08-07 08:53:18,973 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-08-07 08:53:18,973 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=261180939175fe040900ce2fa993a79f, ASSIGN in 373 msec 2023-08-07 08:53:18,974 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-07 08:53:18,975 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398398975"}]},"ts":"1691398398975"} 2023-08-07 08:53:18,977 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-08-07 08:53:18,980 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-08-07 08:53:18,983 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 555 msec 2023-08-07 08:53:19,036 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-08-07 08:53:19,038 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-08-07 08:53:19,038 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 08:53:19,061 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 08:53:19,064 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54386, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-07 08:53:19,081 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-08-07 08:53:19,099 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-07 08:53:19,106 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 35 msec 2023-08-07 08:53:19,115 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:53:19,126 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-07 08:53:19,131 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 15 msec 2023-08-07 08:53:19,142 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-08-07 08:53:19,144 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-08-07 08:53:19,144 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 2.611sec 2023-08-07 08:53:19,147 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-08-07 08:53:19,149 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-08-07 08:53:19,149 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-08-07 08:53:19,150 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33357,1691398395153-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-08-07 08:53:19,150 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33357,1691398395153-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-08-07 08:53:19,159 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-08-07 08:53:19,204 DEBUG [Listener at localhost/33561] zookeeper.ReadOnlyZKClient(139): Connect 0x648b6fc0 to 127.0.0.1:57521 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-07 08:53:19,211 DEBUG [Listener at localhost/33561] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@623ed240, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 08:53:19,225 DEBUG [hconnection-0x36ae5be-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 08:53:19,237 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45148, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-07 08:53:19,246 INFO [Listener at localhost/33561] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,33357,1691398395153 2023-08-07 08:53:19,277 INFO [Listener at localhost/33561] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=360, OpenFileDescriptor=700, MaxFileDescriptor=60000, SystemLoadAverage=336, ProcessCount=174, AvailableMemoryMB=2083 2023-08-07 08:53:19,282 WARN [Listener at localhost/33561] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-07 08:53:19,291 DEBUG [Listener at localhost/33561] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-08-07 08:53:19,294 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48412, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-08-07 08:53:19,307 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-08-07 08:53:19,307 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 08:53:19,308 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-08-07 08:53:19,314 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:19,316 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-08-07 08:53:19,318 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-07 08:53:19,321 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:19,322 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf empty. 2023-08-07 08:53:19,324 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:19,324 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-08-07 08:53:19,346 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-08-07 08:53:19,348 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => 769ddd8aa61f13dd3dc139c4fa3762cf, NAME => 'testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp 2023-08-07 08:53:19,372 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:53:19,373 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1604): Closing 769ddd8aa61f13dd3dc139c4fa3762cf, disabling compactions & flushes 2023-08-07 08:53:19,373 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf. 2023-08-07 08:53:19,373 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf. 2023-08-07 08:53:19,373 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf. after waiting 0 ms 2023-08-07 08:53:19,373 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf. 2023-08-07 08:53:19,373 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf. 2023-08-07 08:53:19,373 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1558): Region close journal for 769ddd8aa61f13dd3dc139c4fa3762cf: 2023-08-07 08:53:19,378 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-08-07 08:53:19,380 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1691398399380"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398399380"}]},"ts":"1691398399380"} 2023-08-07 08:53:19,383 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-07 08:53:19,384 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-07 08:53:19,385 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398399385"}]},"ts":"1691398399385"} 2023-08-07 08:53:19,387 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-08-07 08:53:19,392 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-07 08:53:19,393 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-07 08:53:19,393 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-07 08:53:19,393 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-07 08:53:19,394 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-07 08:53:19,394 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=769ddd8aa61f13dd3dc139c4fa3762cf, ASSIGN}] 2023-08-07 08:53:19,397 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=769ddd8aa61f13dd3dc139c4fa3762cf, ASSIGN 2023-08-07 08:53:19,398 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=769ddd8aa61f13dd3dc139c4fa3762cf, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46525,1691398396377; forceNewPlan=false, retain=false 2023-08-07 08:53:19,548 INFO [jenkins-hbase4:33357] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-07 08:53:19,549 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=769ddd8aa61f13dd3dc139c4fa3762cf, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:19,550 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1691398399549"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398399549"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398399549"}]},"ts":"1691398399549"} 2023-08-07 08:53:19,553 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 769ddd8aa61f13dd3dc139c4fa3762cf, server=jenkins-hbase4.apache.org,46525,1691398396377}] 2023-08-07 08:53:19,713 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf. 2023-08-07 08:53:19,713 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 769ddd8aa61f13dd3dc139c4fa3762cf, NAME => 'testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf.', STARTKEY => '', ENDKEY => ''} 2023-08-07 08:53:19,713 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-07 08:53:19,714 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:19,714 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:53:19,714 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:19,714 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:19,716 INFO [StoreOpener-769ddd8aa61f13dd3dc139c4fa3762cf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:19,718 DEBUG [StoreOpener-769ddd8aa61f13dd3dc139c4fa3762cf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1 2023-08-07 08:53:19,718 DEBUG [StoreOpener-769ddd8aa61f13dd3dc139c4fa3762cf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1 2023-08-07 08:53:19,719 INFO [StoreOpener-769ddd8aa61f13dd3dc139c4fa3762cf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 769ddd8aa61f13dd3dc139c4fa3762cf columnFamilyName cf1 2023-08-07 08:53:19,719 INFO [StoreOpener-769ddd8aa61f13dd3dc139c4fa3762cf-1] regionserver.HStore(310): Store=769ddd8aa61f13dd3dc139c4fa3762cf/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:19,720 INFO [StoreOpener-769ddd8aa61f13dd3dc139c4fa3762cf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:19,722 DEBUG [StoreOpener-769ddd8aa61f13dd3dc139c4fa3762cf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2 2023-08-07 08:53:19,722 DEBUG [StoreOpener-769ddd8aa61f13dd3dc139c4fa3762cf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2 2023-08-07 08:53:19,722 INFO [StoreOpener-769ddd8aa61f13dd3dc139c4fa3762cf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 769ddd8aa61f13dd3dc139c4fa3762cf columnFamilyName cf2 2023-08-07 08:53:19,723 INFO [StoreOpener-769ddd8aa61f13dd3dc139c4fa3762cf-1] regionserver.HStore(310): Store=769ddd8aa61f13dd3dc139c4fa3762cf/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:19,725 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:19,726 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:19,729 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-07 08:53:19,731 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:19,735 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 08:53:19,736 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 769ddd8aa61f13dd3dc139c4fa3762cf; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64306747, jitterRate=-0.04175479710102081}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-07 08:53:19,736 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 769ddd8aa61f13dd3dc139c4fa3762cf: 2023-08-07 08:53:19,738 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf., pid=11, masterSystemTime=1691398399706 2023-08-07 08:53:19,740 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf. 2023-08-07 08:53:19,740 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf. 2023-08-07 08:53:19,742 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=769ddd8aa61f13dd3dc139c4fa3762cf, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:19,742 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1691398399741"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691398399741"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691398399741"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398399741"}]},"ts":"1691398399741"} 2023-08-07 08:53:19,749 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-08-07 08:53:19,749 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 769ddd8aa61f13dd3dc139c4fa3762cf, server=jenkins-hbase4.apache.org,46525,1691398396377 in 192 msec 2023-08-07 08:53:19,752 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-08-07 08:53:19,752 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=769ddd8aa61f13dd3dc139c4fa3762cf, ASSIGN in 355 msec 2023-08-07 08:53:19,753 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-07 08:53:19,754 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398399753"}]},"ts":"1691398399753"} 2023-08-07 08:53:19,756 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-08-07 08:53:19,759 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-08-07 08:53:19,762 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 447 msec 2023-08-07 08:53:19,819 DEBUG [Listener at localhost/33561] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 08:53:19,822 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54398, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-07 08:53:19,842 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:53:19,849 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-08-07 08:53:19,849 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-08-07 08:53:19,849 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:19,850 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-08-07 08:53:19,850 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-08-07 08:53:19,850 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:19,851 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:19,852 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:19,852 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:19,852 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:19,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:19,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:19,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:19,852 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:19,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:19,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:19,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:19,853 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:19,853 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:53:19,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:19,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:19,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:19,853 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:19,853 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:19,854 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:19,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:19,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:19,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:19,854 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-07 08:53:19,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:19,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:19,854 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-07 08:53:19,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:19,854 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-07 08:53:19,856 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-07 08:53:19,856 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-07 08:53:19,856 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-07 08:53:19,856 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-07 08:53:19,856 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-07 08:53:19,856 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-07 08:53:19,856 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:19,856 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:19,857 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-07 08:53:19,857 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-07 08:53:19,856 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:19,857 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-07 08:53:19,858 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-07 08:53:19,858 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-07 08:53:19,858 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-07 08:53:19,858 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-07 08:53:19,859 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:53:19,859 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf. 2023-08-07 08:53:19,859 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:53:19,859 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf. started... 2023-08-07 08:53:19,860 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 769ddd8aa61f13dd3dc139c4fa3762cf 2/2 column families, dataSize=120 B heapSize=928 B 2023-08-07 08:53:19,860 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:19,860 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:19,861 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:19,861 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:19,861 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:19,861 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:19,934 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/.tmp/cf1/659eb35ea67c4594bf1b52bdb4a143f6 2023-08-07 08:53:20,000 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/.tmp/cf2/a65ac280a1f34189883338289cc85639 2023-08-07 08:53:20,010 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/.tmp/cf1/659eb35ea67c4594bf1b52bdb4a143f6 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/659eb35ea67c4594bf1b52bdb4a143f6 2023-08-07 08:53:20,021 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/659eb35ea67c4594bf1b52bdb4a143f6, entries=2, sequenceid=6, filesize=4.8 K 2023-08-07 08:53:20,023 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/.tmp/cf2/a65ac280a1f34189883338289cc85639 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/a65ac280a1f34189883338289cc85639 2023-08-07 08:53:20,032 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/a65ac280a1f34189883338289cc85639, entries=2, sequenceid=6, filesize=4.8 K 2023-08-07 08:53:20,033 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 769ddd8aa61f13dd3dc139c4fa3762cf in 173ms, sequenceid=6, compaction requested=false 2023-08-07 08:53:20,035 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-08-07 08:53:20,037 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 769ddd8aa61f13dd3dc139c4fa3762cf: 2023-08-07 08:53:20,037 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf. 2023-08-07 08:53:20,037 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:53:20,037 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:53:20,038 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,038 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-07 08:53:20,038 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-07 08:53:20,041 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,041 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:20,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:20,041 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,041 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:20,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:20,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:20,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:20,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:20,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:20,045 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-08-07 08:53:20,045 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-08-07 08:53:20,045 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@763aeda2[Count = 0] remaining members to acquire global barrier 2023-08-07 08:53:20,045 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,049 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,049 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,049 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,049 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-07 08:53:20,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,050 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-07 08:53:20,050 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,050 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:53:20,050 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:53:20,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,050 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-07 08:53:20,050 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-07 08:53:20,050 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-07 08:53:20,050 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:53:20,050 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-07 08:53:20,050 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:53:20,053 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-07 08:53:20,053 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:20,053 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-07 08:53:20,055 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,055 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-07 08:53:20,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:20,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:20,055 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-07 08:53:20,056 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:20,056 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:20,056 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-07 08:53:20,056 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-07 08:53:20,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:20,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:20,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:20,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:20,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:20,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:20,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:20,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,46525,1691398396377': 2023-08-07 08:53:20,062 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-08-07 08:53:20,062 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-08-07 08:53:20,062 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:53:20,062 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:53:20,062 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,062 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:53:20,065 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,065 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,065 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,065 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:20,065 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:20,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:20,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:20,065 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,065 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:20,066 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:20,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:20,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:20,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:20,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:20,066 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:20,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:20,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:20,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:20,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,067 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:20,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:20,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:20,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,070 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:20,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:20,070 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:20,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,071 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:20,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:20,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,084 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,084 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,085 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:20,085 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:20,085 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,085 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:20,085 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:20,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:20,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:20,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:20,085 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,085 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:20,085 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,085 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:20,085 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:20,085 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-08-07 08:53:20,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:20,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:20,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:20,085 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:20,085 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:53:20,085 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:20,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:20,085 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:20,085 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:20,086 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:20,087 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,089 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-08-07 08:53:20,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:20,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:20,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:20,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:20,089 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:53:20,189 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:53:20,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:53:20,221 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:53:20,224 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-08-07 08:53:20,225 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,225 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-08-07 08:53:20,225 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:20,225 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-08-07 08:53:20,226 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-08-07 08:53:20,226 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,226 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,228 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:20,228 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:20,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:20,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:20,228 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,228 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:20,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:20,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:20,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:20,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:20,229 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,229 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:53:20,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,229 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,229 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,229 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:20,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:20,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,230 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:20,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,230 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,230 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,230 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-07 08:53:20,230 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,230 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-07 08:53:20,230 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-07 08:53:20,230 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:20,230 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,230 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-07 08:53:20,231 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-07 08:53:20,232 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:20,231 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-07 08:53:20,232 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-07 08:53:20,231 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,232 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-07 08:53:20,232 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:53:20,232 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf. 2023-08-07 08:53:20,233 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:53:20,232 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-07 08:53:20,232 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-07 08:53:20,233 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf. started... 2023-08-07 08:53:20,233 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:20,233 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-07 08:53:20,233 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-07 08:53:20,233 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 769ddd8aa61f13dd3dc139c4fa3762cf 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:53:20,233 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-07 08:53:20,237 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,238 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,238 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,238 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:20,239 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,239 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:20,258 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/.tmp/cf1/3eb4d4087d584c7f9627f94b7ee4146b 2023-08-07 08:53:20,283 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/.tmp/cf2/7825966e17d24694be7beb433a3cfa43 2023-08-07 08:53:20,298 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/.tmp/cf1/3eb4d4087d584c7f9627f94b7ee4146b as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/3eb4d4087d584c7f9627f94b7ee4146b 2023-08-07 08:53:20,307 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/3eb4d4087d584c7f9627f94b7ee4146b, entries=10, sequenceid=19, filesize=5.0 K 2023-08-07 08:53:20,309 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/.tmp/cf2/7825966e17d24694be7beb433a3cfa43 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/7825966e17d24694be7beb433a3cfa43 2023-08-07 08:53:20,318 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/7825966e17d24694be7beb433a3cfa43, entries=10, sequenceid=19, filesize=5.0 K 2023-08-07 08:53:20,319 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 769ddd8aa61f13dd3dc139c4fa3762cf in 86ms, sequenceid=19, compaction requested=false 2023-08-07 08:53:20,319 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 769ddd8aa61f13dd3dc139c4fa3762cf: 2023-08-07 08:53:20,319 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf. 2023-08-07 08:53:20,320 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:53:20,320 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:53:20,320 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,320 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-07 08:53:20,320 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-07 08:53:20,322 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,322 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:20,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:20,323 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,323 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:20,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:20,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:20,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:20,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:20,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:20,326 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-08-07 08:53:20,326 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@32cf15b2[Count = 0] remaining members to acquire global barrier 2023-08-07 08:53:20,326 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-08-07 08:53:20,326 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,327 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,327 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,327 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,328 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-07 08:53:20,328 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-07 08:53:20,328 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-07 08:53:20,328 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-07 08:53:20,328 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,328 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:53:20,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,328 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:53:20,328 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-07 08:53:20,328 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-07 08:53:20,328 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:53:20,328 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:53:20,331 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-07 08:53:20,331 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:20,331 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-07 08:53:20,332 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,332 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-07 08:53:20,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,332 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:20,332 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-07 08:53:20,332 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-07 08:53:20,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:20,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:20,333 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:20,333 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-07 08:53:20,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:20,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:20,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:20,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:20,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:20,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:20,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:20,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,46525,1691398396377': 2023-08-07 08:53:20,338 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-08-07 08:53:20,338 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-08-07 08:53:20,338 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:53:20,338 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:53:20,338 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,338 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:53:20,340 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,340 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,340 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:20,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,340 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:20,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,340 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,340 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:20,340 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:20,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:20,341 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:20,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:20,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:20,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:20,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:20,341 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:20,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:20,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:20,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:20,342 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:20,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:20,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:20,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,345 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:20,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:20,345 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:20,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,346 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:20,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:20,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,362 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,362 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:20,362 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:20,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:20,362 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,362 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:20,362 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:20,362 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:20,362 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:20,362 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-08-07 08:53:20,363 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:53:20,362 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:20,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:20,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:20,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:20,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:20,362 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:20,362 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:20,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:20,362 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,364 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-08-07 08:53:20,364 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:53:20,364 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,364 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:20,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:20,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:20,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:20,364 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:20,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:20,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:20,365 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:20,365 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,365 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,464 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:53:20,466 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:53:20,489 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:53:20,491 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-08-07 08:53:20,493 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,493 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-08-07 08:53:20,494 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:20,494 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-08-07 08:53:20,494 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-08-07 08:53:20,495 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,495 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,496 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:20,496 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:20,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:20,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:20,496 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:20,496 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:20,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:20,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:20,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:20,497 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,497 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:53:20,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,498 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,498 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,498 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:20,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:20,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,498 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:20,498 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,499 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,499 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-07 08:53:20,499 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,499 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-07 08:53:20,499 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-07 08:53:20,499 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-07 08:53:20,499 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:20,499 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,500 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-07 08:53:20,500 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-07 08:53:20,500 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:20,499 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,500 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-07 08:53:20,500 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-07 08:53:20,500 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-07 08:53:20,500 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:20,500 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-07 08:53:20,501 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-07 08:53:20,501 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-07 08:53:20,500 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-07 08:53:20,501 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:53:20,501 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:53:20,501 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf. 2023-08-07 08:53:20,501 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf. started... 2023-08-07 08:53:20,501 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 769ddd8aa61f13dd3dc139c4fa3762cf 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:53:20,503 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,504 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,504 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,504 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:20,504 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,504 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:20,522 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/.tmp/cf1/d52c1e335593428cb634f8dc4ac90982 2023-08-07 08:53:20,546 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/.tmp/cf2/ba9cb9a643e04291a2e4704d02d6dce1 2023-08-07 08:53:20,555 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/.tmp/cf1/d52c1e335593428cb634f8dc4ac90982 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/d52c1e335593428cb634f8dc4ac90982 2023-08-07 08:53:20,562 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/d52c1e335593428cb634f8dc4ac90982, entries=10, sequenceid=32, filesize=5.0 K 2023-08-07 08:53:20,563 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/.tmp/cf2/ba9cb9a643e04291a2e4704d02d6dce1 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/ba9cb9a643e04291a2e4704d02d6dce1 2023-08-07 08:53:20,571 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/ba9cb9a643e04291a2e4704d02d6dce1, entries=10, sequenceid=32, filesize=5.0 K 2023-08-07 08:53:20,572 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 769ddd8aa61f13dd3dc139c4fa3762cf in 71ms, sequenceid=32, compaction requested=true 2023-08-07 08:53:20,572 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 769ddd8aa61f13dd3dc139c4fa3762cf: 2023-08-07 08:53:20,572 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf. 2023-08-07 08:53:20,572 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:53:20,573 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:53:20,573 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,573 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-07 08:53:20,573 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-07 08:53:20,575 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,575 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:20,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:20,576 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,576 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:20,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:20,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:20,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:20,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:20,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:20,578 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-08-07 08:53:20,578 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-08-07 08:53:20,578 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6a0cfeed[Count = 0] remaining members to acquire global barrier 2023-08-07 08:53:20,578 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,580 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,580 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,580 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,580 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-07 08:53:20,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,580 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-07 08:53:20,580 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,580 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:53:20,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,580 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:53:20,580 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-07 08:53:20,580 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-07 08:53:20,580 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:53:20,580 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-07 08:53:20,581 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-07 08:53:20,581 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:53:20,583 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-07 08:53:20,584 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:20,584 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-07 08:53:20,584 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-07 08:53:20,584 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,584 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-07 08:53:20,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,584 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:20,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:20,584 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:20,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:20,585 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-07 08:53:20,586 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-07 08:53:20,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:20,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:20,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:20,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:20,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:20,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:20,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:20,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,46525,1691398396377': 2023-08-07 08:53:20,591 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-08-07 08:53:20,591 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-08-07 08:53:20,591 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:53:20,591 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:53:20,591 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,591 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:53:20,594 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,594 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,594 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,594 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,594 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:20,594 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:20,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:20,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:20,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,594 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:20,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:20,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:20,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:20,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:20,595 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:20,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:20,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:20,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:20,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,596 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:20,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:20,597 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:20,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:20,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:20,600 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:20,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,601 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:20,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:20,601 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:20,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,605 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,605 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,605 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:20,605 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:20,605 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,605 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,605 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:20,605 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:20,605 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:20,605 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:20,605 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:20,606 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:20,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:20,606 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-08-07 08:53:20,606 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:53:20,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:20,606 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:20,606 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:20,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:20,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:20,605 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:20,606 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:20,605 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:20,606 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-08-07 08:53:20,607 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:53:20,606 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:20,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:20,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:20,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:20,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:20,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:20,707 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:53:20,708 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:53:20,732 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:53:20,734 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-08-07 08:53:20,734 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,734 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-08-07 08:53:20,734 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:20,735 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-08-07 08:53:20,735 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-08-07 08:53:20,736 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,736 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,737 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:20,737 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:20,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:20,737 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,737 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:20,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:20,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:20,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:20,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:20,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:20,738 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,738 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:53:20,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,739 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,739 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,739 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:20,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:20,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:20,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,739 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,739 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,739 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,740 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-07 08:53:20,740 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-07 08:53:20,740 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-07 08:53:20,740 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,740 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,740 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-07 08:53:20,740 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-07 08:53:20,740 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,740 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:20,740 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-07 08:53:20,741 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-07 08:53:20,741 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:20,740 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:20,742 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:53:20,742 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-07 08:53:20,742 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf. 2023-08-07 08:53:20,742 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:53:20,742 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-07 08:53:20,742 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf. started... 2023-08-07 08:53:20,742 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-07 08:53:20,742 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-07 08:53:20,742 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-07 08:53:20,743 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 769ddd8aa61f13dd3dc139c4fa3762cf 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:53:20,743 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-07 08:53:20,745 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,746 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,746 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,746 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:20,746 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,746 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:20,763 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/.tmp/cf1/de3670c46dac41a0868e3ab198ad27fa 2023-08-07 08:53:20,788 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/.tmp/cf2/39bb5ddbe47c4def9d6d0bbc388d94c3 2023-08-07 08:53:20,797 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/.tmp/cf1/de3670c46dac41a0868e3ab198ad27fa as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/de3670c46dac41a0868e3ab198ad27fa 2023-08-07 08:53:20,804 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/de3670c46dac41a0868e3ab198ad27fa, entries=10, sequenceid=45, filesize=5.0 K 2023-08-07 08:53:20,806 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/.tmp/cf2/39bb5ddbe47c4def9d6d0bbc388d94c3 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/39bb5ddbe47c4def9d6d0bbc388d94c3 2023-08-07 08:53:20,813 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/39bb5ddbe47c4def9d6d0bbc388d94c3, entries=10, sequenceid=45, filesize=5.0 K 2023-08-07 08:53:20,814 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 769ddd8aa61f13dd3dc139c4fa3762cf in 72ms, sequenceid=45, compaction requested=true 2023-08-07 08:53:20,814 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 769ddd8aa61f13dd3dc139c4fa3762cf: 2023-08-07 08:53:20,814 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf. 2023-08-07 08:53:20,814 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:53:20,815 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:53:20,815 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,815 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-07 08:53:20,815 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-07 08:53:20,819 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,819 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:20,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:20,820 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,820 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:20,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:20,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:20,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:20,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:20,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:20,823 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-08-07 08:53:20,823 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1011e68b[Count = 0] remaining members to acquire global barrier 2023-08-07 08:53:20,823 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-08-07 08:53:20,823 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,824 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,824 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,824 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,824 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,825 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-07 08:53:20,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,825 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-07 08:53:20,825 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:53:20,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,825 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:53:20,825 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-07 08:53:20,825 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-07 08:53:20,825 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-07 08:53:20,825 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-07 08:53:20,825 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:53:20,825 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:53:20,827 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-07 08:53:20,827 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:20,827 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-07 08:53:20,828 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-07 08:53:20,828 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,828 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:20,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:20,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:20,828 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-07 08:53:20,829 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-07 08:53:20,829 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:20,829 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-07 08:53:20,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:20,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:20,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:20,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:20,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:20,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:20,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:20,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,46525,1691398396377': 2023-08-07 08:53:20,833 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-08-07 08:53:20,833 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-08-07 08:53:20,833 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:53:20,833 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:53:20,834 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,834 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:53:20,835 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,835 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,835 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:20,835 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,835 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:20,835 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,835 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:20,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,836 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:20,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:20,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:20,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:20,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:20,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:20,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:20,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:20,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:20,836 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:20,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:20,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,837 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:20,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:20,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:20,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,839 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:20,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:20,839 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:20,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,840 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:20,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:20,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,843 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,843 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,843 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:20,843 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:20,843 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:20,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:20,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:20,843 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:20,843 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:20,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:20,843 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-08-07 08:53:20,843 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:53:20,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:20,843 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:20,843 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:20,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:20,843 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:20,843 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:20,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:20,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:20,845 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-08-07 08:53:20,845 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:53:20,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:20,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:20,844 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:20,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:20,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:20,845 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,845 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,845 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:20,845 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:20,845 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,846 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,945 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:53:20,946 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:53:20,969 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:53:20,971 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-08-07 08:53:20,971 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,971 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-08-07 08:53:20,971 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:20,972 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-08-07 08:53:20,972 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-08-07 08:53:20,973 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,973 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,975 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:20,975 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:20,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:20,975 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:20,975 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:20,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:20,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:20,975 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:20,975 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:53:20,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:20,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:20,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,976 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,976 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:20,977 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,977 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:20,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,977 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,977 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-07 08:53:20,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:20,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,977 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,977 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,977 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-07 08:53:20,978 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:20,977 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-07 08:53:20,979 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-07 08:53:20,979 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,978 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-07 08:53:20,979 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-07 08:53:20,979 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,979 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:20,979 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-07 08:53:20,980 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:20,979 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-07 08:53:20,981 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-07 08:53:20,981 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-07 08:53:20,980 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-07 08:53:20,982 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-07 08:53:20,982 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-07 08:53:20,982 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:53:20,982 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:53:20,982 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf. 2023-08-07 08:53:20,982 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf. started... 2023-08-07 08:53:20,982 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 769ddd8aa61f13dd3dc139c4fa3762cf 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:53:20,983 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,984 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,984 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,984 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:20,985 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:20,985 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:21,005 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/.tmp/cf1/932e1f93301447569e60673ba85254a8 2023-08-07 08:53:21,034 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/.tmp/cf2/36228246fe7340d49afaf2d2784d8676 2023-08-07 08:53:21,042 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/.tmp/cf1/932e1f93301447569e60673ba85254a8 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/932e1f93301447569e60673ba85254a8 2023-08-07 08:53:21,049 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/932e1f93301447569e60673ba85254a8, entries=10, sequenceid=58, filesize=5.0 K 2023-08-07 08:53:21,050 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/.tmp/cf2/36228246fe7340d49afaf2d2784d8676 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/36228246fe7340d49afaf2d2784d8676 2023-08-07 08:53:21,058 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/36228246fe7340d49afaf2d2784d8676, entries=10, sequenceid=58, filesize=5.0 K 2023-08-07 08:53:21,060 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 769ddd8aa61f13dd3dc139c4fa3762cf in 77ms, sequenceid=58, compaction requested=true 2023-08-07 08:53:21,060 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 769ddd8aa61f13dd3dc139c4fa3762cf: 2023-08-07 08:53:21,060 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf. 2023-08-07 08:53:21,060 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:53:21,062 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:53:21,062 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:21,062 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-07 08:53:21,062 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-07 08:53:21,064 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:21,064 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:21,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:21,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:21,065 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,065 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:21,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:21,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:21,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:21,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:21,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:21,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:21,068 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-08-07 08:53:21,068 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-08-07 08:53:21,068 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@31675ed6[Count = 0] remaining members to acquire global barrier 2023-08-07 08:53:21,068 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,071 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,071 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,071 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,071 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:21,071 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:53:21,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,072 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-07 08:53:21,072 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-07 08:53:21,072 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-07 08:53:21,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,072 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:53:21,072 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-07 08:53:21,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,072 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:53:21,072 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-07 08:53:21,072 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-07 08:53:21,072 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:53:21,073 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-07 08:53:21,073 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:21,073 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-07 08:53:21,074 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-07 08:53:21,074 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:21,074 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:21,074 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-07 08:53:21,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:21,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:21,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:21,074 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-07 08:53:21,075 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:21,075 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-07 08:53:21,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:21,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:21,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:21,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:21,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:21,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:21,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:21,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:21,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:21,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,46525,1691398396377': 2023-08-07 08:53:21,079 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-08-07 08:53:21,079 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-08-07 08:53:21,079 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:53:21,079 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:53:21,079 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,079 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:53:21,081 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,081 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,081 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:21,081 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:21,081 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,081 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,081 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:21,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:21,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:21,082 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:21,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:21,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:21,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:21,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:21,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:21,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:21,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:21,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,083 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:21,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,083 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:21,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:21,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:21,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:21,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:21,085 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:21,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:21,086 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:21,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,086 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:21,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:21,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:21,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:21,093 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:21,093 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:21,093 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:21,093 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:21,093 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:21,093 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:21,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:21,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:21,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:21,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:21,093 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-08-07 08:53:21,094 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:53:21,093 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:21,093 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:21,093 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:21,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:21,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:21,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:21,093 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:21,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:21,095 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-08-07 08:53:21,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:21,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:21,095 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:21,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:21,095 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:53:21,095 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,096 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:21,096 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,096 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:21,096 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:21,096 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,096 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,196 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:53:21,197 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:53:21,220 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:53:21,222 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-08-07 08:53:21,222 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,222 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-08-07 08:53:21,222 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:21,223 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-08-07 08:53:21,224 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-08-07 08:53:21,224 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,224 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,225 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:21,226 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:21,226 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:21,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:21,226 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:21,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:21,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:21,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:21,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:21,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:21,226 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:21,226 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:53:21,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,227 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,227 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,227 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:21,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:21,227 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:21,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,227 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,228 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-07 08:53:21,228 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,228 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,228 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-07 08:53:21,228 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-07 08:53:21,228 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-07 08:53:21,228 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,228 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:21,228 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-07 08:53:21,228 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,229 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:21,229 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-07 08:53:21,230 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-07 08:53:21,229 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-07 08:53:21,230 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-07 08:53:21,230 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:21,230 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:53:21,230 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-07 08:53:21,231 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-07 08:53:21,230 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-07 08:53:21,231 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-07 08:53:21,231 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf. 2023-08-07 08:53:21,230 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:53:21,231 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf. started... 2023-08-07 08:53:21,231 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 769ddd8aa61f13dd3dc139c4fa3762cf 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:53:21,232 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,232 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,233 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,233 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:21,233 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,233 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:21,251 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/.tmp/cf1/28c8fcac1e0f4c988ae0be5f5140dbcb 2023-08-07 08:53:21,275 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/.tmp/cf2/ba71af67ab9d4117bb3c565efc95c3b0 2023-08-07 08:53:21,284 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/.tmp/cf1/28c8fcac1e0f4c988ae0be5f5140dbcb as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/28c8fcac1e0f4c988ae0be5f5140dbcb 2023-08-07 08:53:21,292 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/28c8fcac1e0f4c988ae0be5f5140dbcb, entries=10, sequenceid=71, filesize=5.0 K 2023-08-07 08:53:21,293 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/.tmp/cf2/ba71af67ab9d4117bb3c565efc95c3b0 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/ba71af67ab9d4117bb3c565efc95c3b0 2023-08-07 08:53:21,301 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/ba71af67ab9d4117bb3c565efc95c3b0, entries=10, sequenceid=71, filesize=5.0 K 2023-08-07 08:53:21,303 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 769ddd8aa61f13dd3dc139c4fa3762cf in 72ms, sequenceid=71, compaction requested=true 2023-08-07 08:53:21,303 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 769ddd8aa61f13dd3dc139c4fa3762cf: 2023-08-07 08:53:21,303 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf. 2023-08-07 08:53:21,303 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:53:21,304 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:53:21,304 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:21,304 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-07 08:53:21,304 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-07 08:53:21,306 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:21,306 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:21,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:21,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:21,306 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,307 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:21,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:21,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:21,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:21,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:21,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:21,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:21,309 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-08-07 08:53:21,309 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7b4fafcd[Count = 0] remaining members to acquire global barrier 2023-08-07 08:53:21,309 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-08-07 08:53:21,309 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,311 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,311 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,311 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,311 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-07 08:53:21,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,312 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-07 08:53:21,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,312 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:21,312 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:53:21,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,312 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-07 08:53:21,312 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-07 08:53:21,312 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-07 08:53:21,312 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:53:21,312 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:53:21,312 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-07 08:53:21,313 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:53:21,314 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:21,314 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-07 08:53:21,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:21,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:21,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:21,314 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-07 08:53:21,314 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:21,314 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:21,314 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-07 08:53:21,315 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-07 08:53:21,316 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-07 08:53:21,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:21,316 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:21,316 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-07 08:53:21,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:21,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:21,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:21,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:21,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:21,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:21,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:21,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:21,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,46525,1691398396377': 2023-08-07 08:53:21,320 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-08-07 08:53:21,320 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-08-07 08:53:21,320 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:53:21,320 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:53:21,320 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,320 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:53:21,322 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,322 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:21,322 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:21,322 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,322 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,322 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:21,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:21,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,322 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:21,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,323 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:21,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:21,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:21,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:21,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:21,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:21,323 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:21,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:21,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:21,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,324 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:21,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:21,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:21,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:21,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:21,327 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:21,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:21,327 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:21,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,327 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:21,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:21,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:21,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:21,331 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:21,331 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:21,331 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:21,331 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:21,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:21,331 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:21,331 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:21,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:21,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:21,331 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,331 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:21,332 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:21,332 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:21,332 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-08-07 08:53:21,333 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:53:21,332 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:21,332 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:21,332 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:21,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:21,333 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:21,333 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-08-07 08:53:21,333 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:53:21,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:21,333 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:21,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:21,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:21,333 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:21,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:21,334 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:21,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:21,334 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-07 08:53:21,433 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:53:21,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:53:21,456 DEBUG [Listener at localhost/33561] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:21,463 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:53:21,464 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:53:21,464 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:53:21,464 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:53:21,471 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=769ddd8aa61f13dd3dc139c4fa3762cf, UNASSIGN}] 2023-08-07 08:53:21,472 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=769ddd8aa61f13dd3dc139c4fa3762cf, UNASSIGN 2023-08-07 08:53:21,473 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=769ddd8aa61f13dd3dc139c4fa3762cf, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:21,474 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1691398401473"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398401473"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398401473"}]},"ts":"1691398401473"} 2023-08-07 08:53:21,476 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure 769ddd8aa61f13dd3dc139c4fa3762cf, server=jenkins-hbase4.apache.org,46525,1691398396377}] 2023-08-07 08:53:21,636 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:21,637 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 769ddd8aa61f13dd3dc139c4fa3762cf, disabling compactions & flushes 2023-08-07 08:53:21,637 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf. 2023-08-07 08:53:21,637 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf. 2023-08-07 08:53:21,637 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf. after waiting 0 ms 2023-08-07 08:53:21,637 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf. 2023-08-07 08:53:21,637 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 769ddd8aa61f13dd3dc139c4fa3762cf 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-08-07 08:53:21,655 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/.tmp/cf1/4e64012a38374e2ca2434d2f7f29a7ec 2023-08-07 08:53:21,683 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/.tmp/cf2/22bf3f0bba424df3ab01f8dad19ab0ef 2023-08-07 08:53:21,691 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/.tmp/cf1/4e64012a38374e2ca2434d2f7f29a7ec as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/4e64012a38374e2ca2434d2f7f29a7ec 2023-08-07 08:53:21,697 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/4e64012a38374e2ca2434d2f7f29a7ec, entries=8, sequenceid=82, filesize=5.0 K 2023-08-07 08:53:21,699 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/.tmp/cf2/22bf3f0bba424df3ab01f8dad19ab0ef as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/22bf3f0bba424df3ab01f8dad19ab0ef 2023-08-07 08:53:21,708 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/22bf3f0bba424df3ab01f8dad19ab0ef, entries=8, sequenceid=82, filesize=5.0 K 2023-08-07 08:53:21,709 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 769ddd8aa61f13dd3dc139c4fa3762cf in 72ms, sequenceid=82, compaction requested=true 2023-08-07 08:53:21,736 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-08-07 08:53:21,737 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-07 08:53:21,737 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf. 2023-08-07 08:53:21,738 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 769ddd8aa61f13dd3dc139c4fa3762cf: 2023-08-07 08:53:21,740 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:21,741 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=769ddd8aa61f13dd3dc139c4fa3762cf, regionState=CLOSED 2023-08-07 08:53:21,742 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1691398401741"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398401741"}]},"ts":"1691398401741"} 2023-08-07 08:53:21,746 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-08-07 08:53:21,746 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure 769ddd8aa61f13dd3dc139c4fa3762cf, server=jenkins-hbase4.apache.org,46525,1691398396377 in 268 msec 2023-08-07 08:53:21,749 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-08-07 08:53:21,749 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=769ddd8aa61f13dd3dc139c4fa3762cf, UNASSIGN in 275 msec 2023-08-07 08:53:21,770 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=769ddd8aa61f13dd3dc139c4fa3762cf, threads=14 2023-08-07 08:53:21,771 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/22bf3f0bba424df3ab01f8dad19ab0ef for region: 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:21,771 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/36228246fe7340d49afaf2d2784d8676 for region: 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:21,771 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/39bb5ddbe47c4def9d6d0bbc388d94c3 for region: 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:21,771 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/7825966e17d24694be7beb433a3cfa43 for region: 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:21,771 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/a65ac280a1f34189883338289cc85639 for region: 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:21,773 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/ba71af67ab9d4117bb3c565efc95c3b0 for region: 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:21,774 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/ba9cb9a643e04291a2e4704d02d6dce1 for region: 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:21,774 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/28c8fcac1e0f4c988ae0be5f5140dbcb for region: 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:21,774 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/3eb4d4087d584c7f9627f94b7ee4146b for region: 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:21,774 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/659eb35ea67c4594bf1b52bdb4a143f6 for region: 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:21,774 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/4e64012a38374e2ca2434d2f7f29a7ec for region: 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:21,775 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/932e1f93301447569e60673ba85254a8 for region: 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:21,775 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/d52c1e335593428cb634f8dc4ac90982 for region: 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:21,776 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/de3670c46dac41a0868e3ab198ad27fa for region: 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:21,889 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/4e64012a38374e2ca2434d2f7f29a7ec, top=true 2023-08-07 08:53:21,892 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/932e1f93301447569e60673ba85254a8, top=true 2023-08-07 08:53:21,892 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/36228246fe7340d49afaf2d2784d8676, top=true 2023-08-07 08:53:21,919 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/.splits/1d86003e731a97a463196e40fae27785/cf1/testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-4e64012a38374e2ca2434d2f7f29a7ec for child: 1d86003e731a97a463196e40fae27785, parent: 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:21,919 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/4e64012a38374e2ca2434d2f7f29a7ec for region: 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:21,931 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/.splits/1d86003e731a97a463196e40fae27785/cf1/testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-932e1f93301447569e60673ba85254a8 for child: 1d86003e731a97a463196e40fae27785, parent: 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:21,931 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/932e1f93301447569e60673ba85254a8 for region: 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:21,963 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/.splits/1d86003e731a97a463196e40fae27785/cf2/testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-36228246fe7340d49afaf2d2784d8676 for child: 1d86003e731a97a463196e40fae27785, parent: 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:21,964 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/36228246fe7340d49afaf2d2784d8676 for region: 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:21,967 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/22bf3f0bba424df3ab01f8dad19ab0ef, top=true 2023-08-07 08:53:21,968 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/ba71af67ab9d4117bb3c565efc95c3b0, top=true 2023-08-07 08:53:21,972 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/28c8fcac1e0f4c988ae0be5f5140dbcb, top=true 2023-08-07 08:53:22,007 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/a65ac280a1f34189883338289cc85639 for region: 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:22,008 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/.splits/1d86003e731a97a463196e40fae27785/cf2/testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-ba71af67ab9d4117bb3c565efc95c3b0 for child: 1d86003e731a97a463196e40fae27785, parent: 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:22,008 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/ba71af67ab9d4117bb3c565efc95c3b0 for region: 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:22,019 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/.splits/1d86003e731a97a463196e40fae27785/cf2/testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-22bf3f0bba424df3ab01f8dad19ab0ef for child: 1d86003e731a97a463196e40fae27785, parent: 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:22,019 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/22bf3f0bba424df3ab01f8dad19ab0ef for region: 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:22,019 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/.splits/1d86003e731a97a463196e40fae27785/cf1/testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-28c8fcac1e0f4c988ae0be5f5140dbcb for child: 1d86003e731a97a463196e40fae27785, parent: 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:22,019 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/28c8fcac1e0f4c988ae0be5f5140dbcb for region: 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:22,047 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/39bb5ddbe47c4def9d6d0bbc388d94c3 for region: 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:22,048 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/d52c1e335593428cb634f8dc4ac90982 for region: 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:22,052 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/659eb35ea67c4594bf1b52bdb4a143f6 for region: 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:22,065 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/de3670c46dac41a0868e3ab198ad27fa for region: 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:22,067 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/7825966e17d24694be7beb433a3cfa43 for region: 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:22,386 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/ba9cb9a643e04291a2e4704d02d6dce1 for region: 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:22,453 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/3eb4d4087d584c7f9627f94b7ee4146b for region: 769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:22,454 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region 769ddd8aa61f13dd3dc139c4fa3762cf Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-08-07 08:53:22,509 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-08-07 08:53:22,511 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-08-07 08:53:22,514 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1691398402513"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1691398402513"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1691398402513"}]},"ts":"1691398402513"} 2023-08-07 08:53:22,514 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1691398401456.46ac7a76f9146bf6f12230cccd2abdee.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1691398402513"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398402513"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398402513"}]},"ts":"1691398402513"} 2023-08-07 08:53:22,514 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1691398402513"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398402513"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398402513"}]},"ts":"1691398402513"} 2023-08-07 08:53:22,563 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=46ac7a76f9146bf6f12230cccd2abdee, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=1d86003e731a97a463196e40fae27785, ASSIGN}] 2023-08-07 08:53:22,564 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=1d86003e731a97a463196e40fae27785, ASSIGN 2023-08-07 08:53:22,565 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=1d86003e731a97a463196e40fae27785, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,46525,1691398396377; forceNewPlan=false, retain=false 2023-08-07 08:53:22,566 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=46ac7a76f9146bf6f12230cccd2abdee, ASSIGN 2023-08-07 08:53:22,567 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=46ac7a76f9146bf6f12230cccd2abdee, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,46525,1691398396377; forceNewPlan=false, retain=false 2023-08-07 08:53:22,715 INFO [jenkins-hbase4:33357] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-08-07 08:53:22,717 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=1d86003e731a97a463196e40fae27785, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:22,717 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1691398402717"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398402717"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398402717"}]},"ts":"1691398402717"} 2023-08-07 08:53:22,720 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=16, state=RUNNABLE; OpenRegionProcedure 1d86003e731a97a463196e40fae27785, server=jenkins-hbase4.apache.org,46525,1691398396377}] 2023-08-07 08:53:22,724 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=46ac7a76f9146bf6f12230cccd2abdee, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:22,724 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1691398401456.46ac7a76f9146bf6f12230cccd2abdee.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1691398402724"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398402724"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398402724"}]},"ts":"1691398402724"} 2023-08-07 08:53:22,726 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=15, state=RUNNABLE; OpenRegionProcedure 46ac7a76f9146bf6f12230cccd2abdee, server=jenkins-hbase4.apache.org,46525,1691398396377}] 2023-08-07 08:53:22,877 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionUnevenDaughter,,1691398401456.46ac7a76f9146bf6f12230cccd2abdee. 2023-08-07 08:53:22,878 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 46ac7a76f9146bf6f12230cccd2abdee, NAME => 'testSplitTableRegionUnevenDaughter,,1691398401456.46ac7a76f9146bf6f12230cccd2abdee.', STARTKEY => '', ENDKEY => '26'} 2023-08-07 08:53:22,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-07 08:53:22,878 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 46ac7a76f9146bf6f12230cccd2abdee 2023-08-07 08:53:22,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1691398401456.46ac7a76f9146bf6f12230cccd2abdee.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:53:22,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 46ac7a76f9146bf6f12230cccd2abdee 2023-08-07 08:53:22,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 46ac7a76f9146bf6f12230cccd2abdee 2023-08-07 08:53:22,880 INFO [StoreOpener-46ac7a76f9146bf6f12230cccd2abdee-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 46ac7a76f9146bf6f12230cccd2abdee 2023-08-07 08:53:22,882 DEBUG [StoreOpener-46ac7a76f9146bf6f12230cccd2abdee-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf1 2023-08-07 08:53:22,882 DEBUG [StoreOpener-46ac7a76f9146bf6f12230cccd2abdee-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf1 2023-08-07 08:53:22,882 INFO [StoreOpener-46ac7a76f9146bf6f12230cccd2abdee-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 46ac7a76f9146bf6f12230cccd2abdee columnFamilyName cf1 2023-08-07 08:53:22,914 DEBUG [StoreOpener-46ac7a76f9146bf6f12230cccd2abdee-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf1/3eb4d4087d584c7f9627f94b7ee4146b.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/3eb4d4087d584c7f9627f94b7ee4146b-bottom 2023-08-07 08:53:22,921 DEBUG [StoreOpener-46ac7a76f9146bf6f12230cccd2abdee-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf1/659eb35ea67c4594bf1b52bdb4a143f6.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/659eb35ea67c4594bf1b52bdb4a143f6-bottom 2023-08-07 08:53:22,929 DEBUG [StoreOpener-46ac7a76f9146bf6f12230cccd2abdee-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf1/d52c1e335593428cb634f8dc4ac90982.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/d52c1e335593428cb634f8dc4ac90982-bottom 2023-08-07 08:53:22,936 DEBUG [StoreOpener-46ac7a76f9146bf6f12230cccd2abdee-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf1/de3670c46dac41a0868e3ab198ad27fa.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/de3670c46dac41a0868e3ab198ad27fa-bottom 2023-08-07 08:53:22,937 INFO [StoreOpener-46ac7a76f9146bf6f12230cccd2abdee-1] regionserver.HStore(310): Store=46ac7a76f9146bf6f12230cccd2abdee/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:22,937 INFO [StoreOpener-46ac7a76f9146bf6f12230cccd2abdee-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 46ac7a76f9146bf6f12230cccd2abdee 2023-08-07 08:53:22,938 DEBUG [StoreOpener-46ac7a76f9146bf6f12230cccd2abdee-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf2 2023-08-07 08:53:22,938 DEBUG [StoreOpener-46ac7a76f9146bf6f12230cccd2abdee-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf2 2023-08-07 08:53:22,938 INFO [StoreOpener-46ac7a76f9146bf6f12230cccd2abdee-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 46ac7a76f9146bf6f12230cccd2abdee columnFamilyName cf2 2023-08-07 08:53:22,955 DEBUG [StoreOpener-46ac7a76f9146bf6f12230cccd2abdee-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf2/39bb5ddbe47c4def9d6d0bbc388d94c3.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/39bb5ddbe47c4def9d6d0bbc388d94c3-bottom 2023-08-07 08:53:22,962 DEBUG [StoreOpener-46ac7a76f9146bf6f12230cccd2abdee-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf2/7825966e17d24694be7beb433a3cfa43.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/7825966e17d24694be7beb433a3cfa43-bottom 2023-08-07 08:53:22,969 DEBUG [StoreOpener-46ac7a76f9146bf6f12230cccd2abdee-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf2/a65ac280a1f34189883338289cc85639.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/a65ac280a1f34189883338289cc85639-bottom 2023-08-07 08:53:22,976 DEBUG [StoreOpener-46ac7a76f9146bf6f12230cccd2abdee-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf2/ba9cb9a643e04291a2e4704d02d6dce1.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/ba9cb9a643e04291a2e4704d02d6dce1-bottom 2023-08-07 08:53:22,976 INFO [StoreOpener-46ac7a76f9146bf6f12230cccd2abdee-1] regionserver.HStore(310): Store=46ac7a76f9146bf6f12230cccd2abdee/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:22,977 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee 2023-08-07 08:53:22,979 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee 2023-08-07 08:53:22,982 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-07 08:53:22,984 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 46ac7a76f9146bf6f12230cccd2abdee 2023-08-07 08:53:22,985 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 46ac7a76f9146bf6f12230cccd2abdee; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69698320, jitterRate=0.03858590126037598}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-07 08:53:22,985 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 46ac7a76f9146bf6f12230cccd2abdee: 2023-08-07 08:53:22,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1691398401456.46ac7a76f9146bf6f12230cccd2abdee., pid=18, masterSystemTime=1691398402872 2023-08-07 08:53:22,988 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:53:22,989 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-07 08:53:22,990 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-08-07 08:53:22,990 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-08-07 08:53:22,992 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1691398401456.46ac7a76f9146bf6f12230cccd2abdee. 2023-08-07 08:53:22,992 INFO [RS:0;jenkins-hbase4:46525-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1691398401456.46ac7a76f9146bf6f12230cccd2abdee. 2023-08-07 08:53:22,992 INFO [RS:0;jenkins-hbase4:46525-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1691398401456.46ac7a76f9146bf6f12230cccd2abdee. 2023-08-07 08:53:22,992 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] regionserver.HStore(1912): 46ac7a76f9146bf6f12230cccd2abdee/cf1 is initiating minor compaction (all files) 2023-08-07 08:53:22,992 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] regionserver.HStore(1912): 46ac7a76f9146bf6f12230cccd2abdee/cf2 is initiating minor compaction (all files) 2023-08-07 08:53:22,992 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionUnevenDaughter,,1691398401456.46ac7a76f9146bf6f12230cccd2abdee. 2023-08-07 08:53:22,992 INFO [RS:0;jenkins-hbase4:46525-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 46ac7a76f9146bf6f12230cccd2abdee/cf2 in testSplitTableRegionUnevenDaughter,,1691398401456.46ac7a76f9146bf6f12230cccd2abdee. 2023-08-07 08:53:22,992 INFO [RS:0;jenkins-hbase4:46525-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 46ac7a76f9146bf6f12230cccd2abdee/cf1 in testSplitTableRegionUnevenDaughter,,1691398401456.46ac7a76f9146bf6f12230cccd2abdee. 2023-08-07 08:53:22,993 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=46ac7a76f9146bf6f12230cccd2abdee, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:22,993 INFO [RS:0;jenkins-hbase4:46525-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf2/a65ac280a1f34189883338289cc85639.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/a65ac280a1f34189883338289cc85639-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf2/7825966e17d24694be7beb433a3cfa43.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/7825966e17d24694be7beb433a3cfa43-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf2/ba9cb9a643e04291a2e4704d02d6dce1.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/ba9cb9a643e04291a2e4704d02d6dce1-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf2/39bb5ddbe47c4def9d6d0bbc388d94c3.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/39bb5ddbe47c4def9d6d0bbc388d94c3-bottom] into tmpdir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/.tmp, totalSize=19.9 K 2023-08-07 08:53:22,993 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1691398401456.46ac7a76f9146bf6f12230cccd2abdee.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1691398402992"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691398402992"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691398402992"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398402992"}]},"ts":"1691398402992"} 2023-08-07 08:53:22,992 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785. 2023-08-07 08:53:22,993 INFO [RS:0;jenkins-hbase4:46525-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf1/659eb35ea67c4594bf1b52bdb4a143f6.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/659eb35ea67c4594bf1b52bdb4a143f6-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf1/3eb4d4087d584c7f9627f94b7ee4146b.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/3eb4d4087d584c7f9627f94b7ee4146b-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf1/d52c1e335593428cb634f8dc4ac90982.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/d52c1e335593428cb634f8dc4ac90982-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf1/de3670c46dac41a0868e3ab198ad27fa.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/de3670c46dac41a0868e3ab198ad27fa-bottom] into tmpdir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/.tmp, totalSize=19.9 K 2023-08-07 08:53:22,995 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1d86003e731a97a463196e40fae27785, NAME => 'testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785.', STARTKEY => '26', ENDKEY => ''} 2023-08-07 08:53:22,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-07 08:53:22,995 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 1d86003e731a97a463196e40fae27785 2023-08-07 08:53:22,995 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:53:22,996 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1d86003e731a97a463196e40fae27785 2023-08-07 08:53:22,996 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1d86003e731a97a463196e40fae27785 2023-08-07 08:53:22,996 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] compactions.Compactor(207): Compacting a65ac280a1f34189883338289cc85639.769ddd8aa61f13dd3dc139c4fa3762cf, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1691398399823 2023-08-07 08:53:22,997 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] compactions.Compactor(207): Compacting 659eb35ea67c4594bf1b52bdb4a143f6.769ddd8aa61f13dd3dc139c4fa3762cf, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1691398399823 2023-08-07 08:53:22,997 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] compactions.Compactor(207): Compacting 7825966e17d24694be7beb433a3cfa43.769ddd8aa61f13dd3dc139c4fa3762cf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1691398400198 2023-08-07 08:53:22,998 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] compactions.Compactor(207): Compacting 3eb4d4087d584c7f9627f94b7ee4146b.769ddd8aa61f13dd3dc139c4fa3762cf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1691398400198 2023-08-07 08:53:22,998 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] compactions.Compactor(207): Compacting ba9cb9a643e04291a2e4704d02d6dce1.769ddd8aa61f13dd3dc139c4fa3762cf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1691398400467 2023-08-07 08:53:22,998 INFO [StoreOpener-1d86003e731a97a463196e40fae27785-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 1d86003e731a97a463196e40fae27785 2023-08-07 08:53:22,998 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] compactions.Compactor(207): Compacting d52c1e335593428cb634f8dc4ac90982.769ddd8aa61f13dd3dc139c4fa3762cf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1691398400467 2023-08-07 08:53:22,999 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] compactions.Compactor(207): Compacting 39bb5ddbe47c4def9d6d0bbc388d94c3.769ddd8aa61f13dd3dc139c4fa3762cf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1691398400710 2023-08-07 08:53:22,999 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] compactions.Compactor(207): Compacting de3670c46dac41a0868e3ab198ad27fa.769ddd8aa61f13dd3dc139c4fa3762cf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1691398400710 2023-08-07 08:53:23,002 DEBUG [StoreOpener-1d86003e731a97a463196e40fae27785-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1 2023-08-07 08:53:23,002 DEBUG [StoreOpener-1d86003e731a97a463196e40fae27785-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1 2023-08-07 08:53:23,002 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=15 2023-08-07 08:53:23,002 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=15, state=SUCCESS; OpenRegionProcedure 46ac7a76f9146bf6f12230cccd2abdee, server=jenkins-hbase4.apache.org,46525,1691398396377 in 272 msec 2023-08-07 08:53:23,002 INFO [StoreOpener-1d86003e731a97a463196e40fae27785-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1d86003e731a97a463196e40fae27785 columnFamilyName cf1 2023-08-07 08:53:23,005 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=46ac7a76f9146bf6f12230cccd2abdee, ASSIGN in 439 msec 2023-08-07 08:53:23,029 INFO [RS:0;jenkins-hbase4:46525-longCompactions-0] throttle.PressureAwareThroughputController(145): 46ac7a76f9146bf6f12230cccd2abdee#cf2#compaction#14 average throughput is 0.21 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-07 08:53:23,033 INFO [RS:0;jenkins-hbase4:46525-shortCompactions-0] throttle.PressureAwareThroughputController(145): 46ac7a76f9146bf6f12230cccd2abdee#cf1#compaction#15 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-07 08:53:23,033 DEBUG [StoreOpener-1d86003e731a97a463196e40fae27785-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/3eb4d4087d584c7f9627f94b7ee4146b.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/3eb4d4087d584c7f9627f94b7ee4146b-top 2023-08-07 08:53:23,040 DEBUG [StoreOpener-1d86003e731a97a463196e40fae27785-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/659eb35ea67c4594bf1b52bdb4a143f6.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/659eb35ea67c4594bf1b52bdb4a143f6-top 2023-08-07 08:53:23,050 DEBUG [StoreOpener-1d86003e731a97a463196e40fae27785-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/d52c1e335593428cb634f8dc4ac90982.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/d52c1e335593428cb634f8dc4ac90982-top 2023-08-07 08:53:23,062 DEBUG [StoreOpener-1d86003e731a97a463196e40fae27785-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/de3670c46dac41a0868e3ab198ad27fa.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/de3670c46dac41a0868e3ab198ad27fa-top 2023-08-07 08:53:23,074 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/.tmp/cf1/8f578aed284e4d74b3e1c8698ba5ab3b as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf1/8f578aed284e4d74b3e1c8698ba5ab3b 2023-08-07 08:53:23,075 DEBUG [StoreOpener-1d86003e731a97a463196e40fae27785-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-28c8fcac1e0f4c988ae0be5f5140dbcb 2023-08-07 08:53:23,078 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/.tmp/cf2/0fc29b2b52b847cba7411c3b6e04901a as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf2/0fc29b2b52b847cba7411c3b6e04901a 2023-08-07 08:53:23,084 DEBUG [StoreOpener-1d86003e731a97a463196e40fae27785-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-4e64012a38374e2ca2434d2f7f29a7ec 2023-08-07 08:53:23,095 INFO [RS:0;jenkins-hbase4:46525-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 46ac7a76f9146bf6f12230cccd2abdee/cf2 of 46ac7a76f9146bf6f12230cccd2abdee into 0fc29b2b52b847cba7411c3b6e04901a(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 08:53:23,095 INFO [RS:0;jenkins-hbase4:46525-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 46ac7a76f9146bf6f12230cccd2abdee/cf1 of 46ac7a76f9146bf6f12230cccd2abdee into 8f578aed284e4d74b3e1c8698ba5ab3b(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 08:53:23,099 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 46ac7a76f9146bf6f12230cccd2abdee: 2023-08-07 08:53:23,099 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 46ac7a76f9146bf6f12230cccd2abdee: 2023-08-07 08:53:23,099 INFO [RS:0;jenkins-hbase4:46525-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1691398401456.46ac7a76f9146bf6f12230cccd2abdee., storeName=46ac7a76f9146bf6f12230cccd2abdee/cf1, priority=12, startTime=1691398402988; duration=0sec 2023-08-07 08:53:23,099 INFO [RS:0;jenkins-hbase4:46525-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1691398401456.46ac7a76f9146bf6f12230cccd2abdee., storeName=46ac7a76f9146bf6f12230cccd2abdee/cf2, priority=12, startTime=1691398402989; duration=0sec 2023-08-07 08:53:23,100 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:53:23,100 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:53:23,100 DEBUG [StoreOpener-1d86003e731a97a463196e40fae27785-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-932e1f93301447569e60673ba85254a8 2023-08-07 08:53:23,101 INFO [StoreOpener-1d86003e731a97a463196e40fae27785-1] regionserver.HStore(310): Store=1d86003e731a97a463196e40fae27785/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:23,101 INFO [StoreOpener-1d86003e731a97a463196e40fae27785-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 1d86003e731a97a463196e40fae27785 2023-08-07 08:53:23,102 DEBUG [StoreOpener-1d86003e731a97a463196e40fae27785-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2 2023-08-07 08:53:23,102 DEBUG [StoreOpener-1d86003e731a97a463196e40fae27785-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2 2023-08-07 08:53:23,103 INFO [StoreOpener-1d86003e731a97a463196e40fae27785-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1d86003e731a97a463196e40fae27785 columnFamilyName cf2 2023-08-07 08:53:23,119 DEBUG [StoreOpener-1d86003e731a97a463196e40fae27785-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/39bb5ddbe47c4def9d6d0bbc388d94c3.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/39bb5ddbe47c4def9d6d0bbc388d94c3-top 2023-08-07 08:53:23,125 DEBUG [StoreOpener-1d86003e731a97a463196e40fae27785-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/7825966e17d24694be7beb433a3cfa43.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/7825966e17d24694be7beb433a3cfa43-top 2023-08-07 08:53:23,130 DEBUG [StoreOpener-1d86003e731a97a463196e40fae27785-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/a65ac280a1f34189883338289cc85639.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/a65ac280a1f34189883338289cc85639-top 2023-08-07 08:53:23,135 DEBUG [StoreOpener-1d86003e731a97a463196e40fae27785-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/ba9cb9a643e04291a2e4704d02d6dce1.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/ba9cb9a643e04291a2e4704d02d6dce1-top 2023-08-07 08:53:23,142 DEBUG [StoreOpener-1d86003e731a97a463196e40fae27785-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-22bf3f0bba424df3ab01f8dad19ab0ef 2023-08-07 08:53:23,147 DEBUG [StoreOpener-1d86003e731a97a463196e40fae27785-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-36228246fe7340d49afaf2d2784d8676 2023-08-07 08:53:23,151 DEBUG [StoreOpener-1d86003e731a97a463196e40fae27785-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-ba71af67ab9d4117bb3c565efc95c3b0 2023-08-07 08:53:23,152 INFO [StoreOpener-1d86003e731a97a463196e40fae27785-1] regionserver.HStore(310): Store=1d86003e731a97a463196e40fae27785/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:23,152 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785 2023-08-07 08:53:23,155 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785 2023-08-07 08:53:23,157 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-07 08:53:23,158 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1d86003e731a97a463196e40fae27785 2023-08-07 08:53:23,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1d86003e731a97a463196e40fae27785; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65594405, jitterRate=-0.022567197680473328}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-07 08:53:23,160 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1d86003e731a97a463196e40fae27785: 2023-08-07 08:53:23,161 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785., pid=17, masterSystemTime=1691398402872 2023-08-07 08:53:23,161 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-07 08:53:23,162 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-07 08:53:23,162 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-07 08:53:23,162 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:53:23,162 INFO [RS:0;jenkins-hbase4:46525-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785. 2023-08-07 08:53:23,163 INFO [RS:0;jenkins-hbase4:46525-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785. 2023-08-07 08:53:23,163 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] regionserver.HStore(1912): 1d86003e731a97a463196e40fae27785/cf2 is initiating minor compaction (all files) 2023-08-07 08:53:23,163 INFO [RS:0;jenkins-hbase4:46525-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 1d86003e731a97a463196e40fae27785/cf2 in testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785. 2023-08-07 08:53:23,163 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] regionserver.HStore(1912): 1d86003e731a97a463196e40fae27785/cf1 is initiating minor compaction (all files) 2023-08-07 08:53:23,163 INFO [RS:0;jenkins-hbase4:46525-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 1d86003e731a97a463196e40fae27785/cf1 in testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785. 2023-08-07 08:53:23,164 INFO [RS:0;jenkins-hbase4:46525-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/a65ac280a1f34189883338289cc85639.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/a65ac280a1f34189883338289cc85639-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/7825966e17d24694be7beb433a3cfa43.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/7825966e17d24694be7beb433a3cfa43-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/ba9cb9a643e04291a2e4704d02d6dce1.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/ba9cb9a643e04291a2e4704d02d6dce1-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/39bb5ddbe47c4def9d6d0bbc388d94c3.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/39bb5ddbe47c4def9d6d0bbc388d94c3-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-36228246fe7340d49afaf2d2784d8676, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-ba71af67ab9d4117bb3c565efc95c3b0, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-22bf3f0bba424df3ab01f8dad19ab0ef] into tmpdir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/.tmp, totalSize=34.9 K 2023-08-07 08:53:23,164 INFO [RS:0;jenkins-hbase4:46525-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/659eb35ea67c4594bf1b52bdb4a143f6.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/659eb35ea67c4594bf1b52bdb4a143f6-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/3eb4d4087d584c7f9627f94b7ee4146b.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/3eb4d4087d584c7f9627f94b7ee4146b-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/d52c1e335593428cb634f8dc4ac90982.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/d52c1e335593428cb634f8dc4ac90982-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/de3670c46dac41a0868e3ab198ad27fa.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/de3670c46dac41a0868e3ab198ad27fa-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-932e1f93301447569e60673ba85254a8, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-28c8fcac1e0f4c988ae0be5f5140dbcb, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-4e64012a38374e2ca2434d2f7f29a7ec] into tmpdir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/.tmp, totalSize=34.9 K 2023-08-07 08:53:23,165 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] compactions.Compactor(207): Compacting a65ac280a1f34189883338289cc85639.769ddd8aa61f13dd3dc139c4fa3762cf, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1691398399823 2023-08-07 08:53:23,166 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785. 2023-08-07 08:53:23,166 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] compactions.Compactor(207): Compacting 659eb35ea67c4594bf1b52bdb4a143f6.769ddd8aa61f13dd3dc139c4fa3762cf, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1691398399823 2023-08-07 08:53:23,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785. 2023-08-07 08:53:23,168 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=1d86003e731a97a463196e40fae27785, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:23,168 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] compactions.Compactor(207): Compacting 7825966e17d24694be7beb433a3cfa43.769ddd8aa61f13dd3dc139c4fa3762cf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1691398400198 2023-08-07 08:53:23,168 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1691398403168"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691398403168"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691398403168"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398403168"}]},"ts":"1691398403168"} 2023-08-07 08:53:23,169 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] compactions.Compactor(207): Compacting 3eb4d4087d584c7f9627f94b7ee4146b.769ddd8aa61f13dd3dc139c4fa3762cf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1691398400198 2023-08-07 08:53:23,169 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] compactions.Compactor(207): Compacting ba9cb9a643e04291a2e4704d02d6dce1.769ddd8aa61f13dd3dc139c4fa3762cf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1691398400467 2023-08-07 08:53:23,171 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] compactions.Compactor(207): Compacting d52c1e335593428cb634f8dc4ac90982.769ddd8aa61f13dd3dc139c4fa3762cf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1691398400467 2023-08-07 08:53:23,171 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] compactions.Compactor(207): Compacting 39bb5ddbe47c4def9d6d0bbc388d94c3.769ddd8aa61f13dd3dc139c4fa3762cf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1691398400710 2023-08-07 08:53:23,171 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] compactions.Compactor(207): Compacting de3670c46dac41a0868e3ab198ad27fa.769ddd8aa61f13dd3dc139c4fa3762cf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1691398400710 2023-08-07 08:53:23,172 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-36228246fe7340d49afaf2d2784d8676, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1691398400948 2023-08-07 08:53:23,172 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-932e1f93301447569e60673ba85254a8, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1691398400948 2023-08-07 08:53:23,173 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-ba71af67ab9d4117bb3c565efc95c3b0, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1691398401198 2023-08-07 08:53:23,174 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-28c8fcac1e0f4c988ae0be5f5140dbcb, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1691398401198 2023-08-07 08:53:23,174 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-22bf3f0bba424df3ab01f8dad19ab0ef, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1691398401436 2023-08-07 08:53:23,175 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-4e64012a38374e2ca2434d2f7f29a7ec, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1691398401436 2023-08-07 08:53:23,176 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=16 2023-08-07 08:53:23,176 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=16, state=SUCCESS; OpenRegionProcedure 1d86003e731a97a463196e40fae27785, server=jenkins-hbase4.apache.org,46525,1691398396377 in 452 msec 2023-08-07 08:53:23,180 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=12 2023-08-07 08:53:23,180 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=1d86003e731a97a463196e40fae27785, ASSIGN in 613 msec 2023-08-07 08:53:23,183 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 in 1.7230 sec 2023-08-07 08:53:23,208 INFO [RS:0;jenkins-hbase4:46525-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1d86003e731a97a463196e40fae27785#cf2#compaction#16 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-07 08:53:23,210 INFO [RS:0;jenkins-hbase4:46525-longCompactions-0] throttle.PressureAwareThroughputController(145): 1d86003e731a97a463196e40fae27785#cf1#compaction#17 average throughput is 0.64 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-07 08:53:23,224 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46525] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:54398 deadline: 1691398463221, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf. is not online on jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:23,261 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/.tmp/cf1/4edb992b5fa2457196c94ff7a6029eaf as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/4edb992b5fa2457196c94ff7a6029eaf 2023-08-07 08:53:23,262 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/.tmp/cf2/87b571fd8e7c468ab048a070d07e906f as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/87b571fd8e7c468ab048a070d07e906f 2023-08-07 08:53:23,269 INFO [RS:0;jenkins-hbase4:46525-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 1d86003e731a97a463196e40fae27785/cf1 of 1d86003e731a97a463196e40fae27785 into 4edb992b5fa2457196c94ff7a6029eaf(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 08:53:23,270 INFO [RS:0;jenkins-hbase4:46525-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 1d86003e731a97a463196e40fae27785/cf2 of 1d86003e731a97a463196e40fae27785 into 87b571fd8e7c468ab048a070d07e906f(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 08:53:23,271 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1d86003e731a97a463196e40fae27785: 2023-08-07 08:53:23,271 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1d86003e731a97a463196e40fae27785: 2023-08-07 08:53:23,271 INFO [RS:0;jenkins-hbase4:46525-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785., storeName=1d86003e731a97a463196e40fae27785/cf1, priority=9, startTime=1691398403161; duration=0sec 2023-08-07 08:53:23,272 INFO [RS:0;jenkins-hbase4:46525-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785., storeName=1d86003e731a97a463196e40fae27785/cf2, priority=9, startTime=1691398403162; duration=0sec 2023-08-07 08:53:23,272 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:53:23,272 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:53:23,386 DEBUG [Listener at localhost/33561] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-08-07 08:53:23,397 INFO [Listener at localhost/33561] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-08-07 08:53:23,400 DEBUG [Listener at localhost/33561] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-08-07 08:53:23,434 WARN [Listener at localhost/33561] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-07 08:53:23,446 INFO [Listener at localhost/33561] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-08-07 08:53:23,450 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionUnevenDaughter 2023-08-07 08:53:23,460 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:23,467 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398403467"}]},"ts":"1691398403467"} 2023-08-07 08:53:23,469 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-08-07 08:53:23,472 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-08-07 08:53:23,473 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-08-07 08:53:23,474 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=46ac7a76f9146bf6f12230cccd2abdee, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=1d86003e731a97a463196e40fae27785, UNASSIGN}] 2023-08-07 08:53:23,475 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=1d86003e731a97a463196e40fae27785, UNASSIGN 2023-08-07 08:53:23,477 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=1d86003e731a97a463196e40fae27785, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:23,477 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1691398403476"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398403476"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398403476"}]},"ts":"1691398403476"} 2023-08-07 08:53:23,479 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure 1d86003e731a97a463196e40fae27785, server=jenkins-hbase4.apache.org,46525,1691398396377}] 2023-08-07 08:53:23,483 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=46ac7a76f9146bf6f12230cccd2abdee, UNASSIGN 2023-08-07 08:53:23,485 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=46ac7a76f9146bf6f12230cccd2abdee, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:23,485 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1691398401456.46ac7a76f9146bf6f12230cccd2abdee.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1691398403485"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398403485"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398403485"}]},"ts":"1691398403485"} 2023-08-07 08:53:23,487 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure 46ac7a76f9146bf6f12230cccd2abdee, server=jenkins-hbase4.apache.org,46525,1691398396377}] 2023-08-07 08:53:23,578 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-08-07 08:53:23,632 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 46ac7a76f9146bf6f12230cccd2abdee 2023-08-07 08:53:23,633 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 46ac7a76f9146bf6f12230cccd2abdee, disabling compactions & flushes 2023-08-07 08:53:23,633 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1691398401456.46ac7a76f9146bf6f12230cccd2abdee. 2023-08-07 08:53:23,633 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1691398401456.46ac7a76f9146bf6f12230cccd2abdee. 2023-08-07 08:53:23,633 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1691398401456.46ac7a76f9146bf6f12230cccd2abdee. after waiting 0 ms 2023-08-07 08:53:23,633 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1691398401456.46ac7a76f9146bf6f12230cccd2abdee. 2023-08-07 08:53:23,639 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1691398401456.46ac7a76f9146bf6f12230cccd2abdee.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf1/659eb35ea67c4594bf1b52bdb4a143f6.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/659eb35ea67c4594bf1b52bdb4a143f6-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf1/3eb4d4087d584c7f9627f94b7ee4146b.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/3eb4d4087d584c7f9627f94b7ee4146b-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf1/d52c1e335593428cb634f8dc4ac90982.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/d52c1e335593428cb634f8dc4ac90982-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf1/de3670c46dac41a0868e3ab198ad27fa.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/de3670c46dac41a0868e3ab198ad27fa-bottom] to archive 2023-08-07 08:53:23,641 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1691398401456.46ac7a76f9146bf6f12230cccd2abdee.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-07 08:53:23,647 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1691398401456.46ac7a76f9146bf6f12230cccd2abdee.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf1/659eb35ea67c4594bf1b52bdb4a143f6.769ddd8aa61f13dd3dc139c4fa3762cf to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf1/659eb35ea67c4594bf1b52bdb4a143f6.769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:23,649 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1691398401456.46ac7a76f9146bf6f12230cccd2abdee.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf1/3eb4d4087d584c7f9627f94b7ee4146b.769ddd8aa61f13dd3dc139c4fa3762cf to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf1/3eb4d4087d584c7f9627f94b7ee4146b.769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:23,651 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1691398401456.46ac7a76f9146bf6f12230cccd2abdee.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf1/d52c1e335593428cb634f8dc4ac90982.769ddd8aa61f13dd3dc139c4fa3762cf to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf1/d52c1e335593428cb634f8dc4ac90982.769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:23,653 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1691398401456.46ac7a76f9146bf6f12230cccd2abdee.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf1/de3670c46dac41a0868e3ab198ad27fa.769ddd8aa61f13dd3dc139c4fa3762cf to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf1/de3670c46dac41a0868e3ab198ad27fa.769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:23,681 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1691398401456.46ac7a76f9146bf6f12230cccd2abdee.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf2/a65ac280a1f34189883338289cc85639.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/a65ac280a1f34189883338289cc85639-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf2/7825966e17d24694be7beb433a3cfa43.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/7825966e17d24694be7beb433a3cfa43-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf2/ba9cb9a643e04291a2e4704d02d6dce1.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/ba9cb9a643e04291a2e4704d02d6dce1-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf2/39bb5ddbe47c4def9d6d0bbc388d94c3.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/39bb5ddbe47c4def9d6d0bbc388d94c3-bottom] to archive 2023-08-07 08:53:23,682 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1691398401456.46ac7a76f9146bf6f12230cccd2abdee.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-07 08:53:23,684 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1691398401456.46ac7a76f9146bf6f12230cccd2abdee.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf2/a65ac280a1f34189883338289cc85639.769ddd8aa61f13dd3dc139c4fa3762cf to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf2/a65ac280a1f34189883338289cc85639.769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:23,687 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1691398401456.46ac7a76f9146bf6f12230cccd2abdee.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf2/7825966e17d24694be7beb433a3cfa43.769ddd8aa61f13dd3dc139c4fa3762cf to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf2/7825966e17d24694be7beb433a3cfa43.769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:23,689 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1691398401456.46ac7a76f9146bf6f12230cccd2abdee.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf2/ba9cb9a643e04291a2e4704d02d6dce1.769ddd8aa61f13dd3dc139c4fa3762cf to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf2/ba9cb9a643e04291a2e4704d02d6dce1.769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:23,690 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1691398401456.46ac7a76f9146bf6f12230cccd2abdee.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf2/39bb5ddbe47c4def9d6d0bbc388d94c3.769ddd8aa61f13dd3dc139c4fa3762cf to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf2/39bb5ddbe47c4def9d6d0bbc388d94c3.769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:23,703 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-08-07 08:53:23,704 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-07 08:53:23,704 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1691398401456.46ac7a76f9146bf6f12230cccd2abdee. 2023-08-07 08:53:23,704 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 46ac7a76f9146bf6f12230cccd2abdee: 2023-08-07 08:53:23,706 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 46ac7a76f9146bf6f12230cccd2abdee 2023-08-07 08:53:23,707 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 1d86003e731a97a463196e40fae27785 2023-08-07 08:53:23,707 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1d86003e731a97a463196e40fae27785, disabling compactions & flushes 2023-08-07 08:53:23,707 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785. 2023-08-07 08:53:23,707 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785. 2023-08-07 08:53:23,707 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785. after waiting 0 ms 2023-08-07 08:53:23,707 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785. 2023-08-07 08:53:23,708 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=46ac7a76f9146bf6f12230cccd2abdee, regionState=CLOSED 2023-08-07 08:53:23,710 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1691398401456.46ac7a76f9146bf6f12230cccd2abdee.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1691398403707"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398403707"}]},"ts":"1691398403707"} 2023-08-07 08:53:23,711 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/659eb35ea67c4594bf1b52bdb4a143f6.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/659eb35ea67c4594bf1b52bdb4a143f6-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/3eb4d4087d584c7f9627f94b7ee4146b.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/3eb4d4087d584c7f9627f94b7ee4146b-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/d52c1e335593428cb634f8dc4ac90982.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/d52c1e335593428cb634f8dc4ac90982-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/de3670c46dac41a0868e3ab198ad27fa.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/de3670c46dac41a0868e3ab198ad27fa-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-932e1f93301447569e60673ba85254a8, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-28c8fcac1e0f4c988ae0be5f5140dbcb, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-4e64012a38374e2ca2434d2f7f29a7ec] to archive 2023-08-07 08:53:23,712 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-07 08:53:23,716 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/659eb35ea67c4594bf1b52bdb4a143f6.769ddd8aa61f13dd3dc139c4fa3762cf to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/659eb35ea67c4594bf1b52bdb4a143f6.769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:23,717 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=20 2023-08-07 08:53:23,717 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure 46ac7a76f9146bf6f12230cccd2abdee, server=jenkins-hbase4.apache.org,46525,1691398396377 in 226 msec 2023-08-07 08:53:23,718 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/3eb4d4087d584c7f9627f94b7ee4146b.769ddd8aa61f13dd3dc139c4fa3762cf to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/3eb4d4087d584c7f9627f94b7ee4146b.769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:23,719 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=46ac7a76f9146bf6f12230cccd2abdee, UNASSIGN in 243 msec 2023-08-07 08:53:23,720 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/d52c1e335593428cb634f8dc4ac90982.769ddd8aa61f13dd3dc139c4fa3762cf to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/d52c1e335593428cb634f8dc4ac90982.769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:23,721 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/de3670c46dac41a0868e3ab198ad27fa.769ddd8aa61f13dd3dc139c4fa3762cf to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/de3670c46dac41a0868e3ab198ad27fa.769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:23,723 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-932e1f93301447569e60673ba85254a8 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-932e1f93301447569e60673ba85254a8 2023-08-07 08:53:23,724 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-28c8fcac1e0f4c988ae0be5f5140dbcb to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-28c8fcac1e0f4c988ae0be5f5140dbcb 2023-08-07 08:53:23,726 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-4e64012a38374e2ca2434d2f7f29a7ec to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-4e64012a38374e2ca2434d2f7f29a7ec 2023-08-07 08:53:23,728 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/a65ac280a1f34189883338289cc85639.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/a65ac280a1f34189883338289cc85639-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/7825966e17d24694be7beb433a3cfa43.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/7825966e17d24694be7beb433a3cfa43-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/ba9cb9a643e04291a2e4704d02d6dce1.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/ba9cb9a643e04291a2e4704d02d6dce1-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/39bb5ddbe47c4def9d6d0bbc388d94c3.769ddd8aa61f13dd3dc139c4fa3762cf->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/39bb5ddbe47c4def9d6d0bbc388d94c3-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-36228246fe7340d49afaf2d2784d8676, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-ba71af67ab9d4117bb3c565efc95c3b0, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-22bf3f0bba424df3ab01f8dad19ab0ef] to archive 2023-08-07 08:53:23,729 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-07 08:53:23,731 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/a65ac280a1f34189883338289cc85639.769ddd8aa61f13dd3dc139c4fa3762cf to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/a65ac280a1f34189883338289cc85639.769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:23,733 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/7825966e17d24694be7beb433a3cfa43.769ddd8aa61f13dd3dc139c4fa3762cf to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/7825966e17d24694be7beb433a3cfa43.769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:23,735 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/ba9cb9a643e04291a2e4704d02d6dce1.769ddd8aa61f13dd3dc139c4fa3762cf to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/ba9cb9a643e04291a2e4704d02d6dce1.769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:23,736 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/39bb5ddbe47c4def9d6d0bbc388d94c3.769ddd8aa61f13dd3dc139c4fa3762cf to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/39bb5ddbe47c4def9d6d0bbc388d94c3.769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:23,738 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-36228246fe7340d49afaf2d2784d8676 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-36228246fe7340d49afaf2d2784d8676 2023-08-07 08:53:23,739 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-ba71af67ab9d4117bb3c565efc95c3b0 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-ba71af67ab9d4117bb3c565efc95c3b0 2023-08-07 08:53:23,741 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-22bf3f0bba424df3ab01f8dad19ab0ef to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/testSplitTableRegionUnevenDaughter=769ddd8aa61f13dd3dc139c4fa3762cf-22bf3f0bba424df3ab01f8dad19ab0ef 2023-08-07 08:53:23,746 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-08-07 08:53:23,747 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-07 08:53:23,747 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785. 2023-08-07 08:53:23,747 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1d86003e731a97a463196e40fae27785: 2023-08-07 08:53:23,750 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 1d86003e731a97a463196e40fae27785 2023-08-07 08:53:23,751 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=1d86003e731a97a463196e40fae27785, regionState=CLOSED 2023-08-07 08:53:23,751 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1691398403750"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398403750"}]},"ts":"1691398403750"} 2023-08-07 08:53:23,754 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-08-07 08:53:23,765 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-08-07 08:53:23,766 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure 1d86003e731a97a463196e40fae27785, server=jenkins-hbase4.apache.org,46525,1691398396377 in 274 msec 2023-08-07 08:53:23,770 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=19 2023-08-07 08:53:23,770 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=1d86003e731a97a463196e40fae27785, UNASSIGN in 292 msec 2023-08-07 08:53:23,772 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398403772"}]},"ts":"1691398403772"} 2023-08-07 08:53:23,774 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-08-07 08:53:23,777 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-08-07 08:53:23,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-08-07 08:53:23,783 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 325 msec 2023-08-07 08:53:23,842 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-08-07 08:53:23,843 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-08-07 08:53:24,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-08-07 08:53:24,083 INFO [Listener at localhost/33561] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-08-07 08:53:24,088 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionUnevenDaughter 2023-08-07 08:53:24,094 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:24,096 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:24,099 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:24,099 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=24 2023-08-07 08:53:24,106 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:24,107 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee 2023-08-07 08:53:24,107 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785 2023-08-07 08:53:24,110 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/recovered.edits] 2023-08-07 08:53:24,110 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/recovered.edits] 2023-08-07 08:53:24,110 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf1, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf2, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/recovered.edits] 2023-08-07 08:53:24,116 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/28c8fcac1e0f4c988ae0be5f5140dbcb to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/28c8fcac1e0f4c988ae0be5f5140dbcb 2023-08-07 08:53:24,116 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/4edb992b5fa2457196c94ff7a6029eaf to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf1/4edb992b5fa2457196c94ff7a6029eaf 2023-08-07 08:53:24,116 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf1/8f578aed284e4d74b3e1c8698ba5ab3b to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf1/8f578aed284e4d74b3e1c8698ba5ab3b 2023-08-07 08:53:24,118 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/3eb4d4087d584c7f9627f94b7ee4146b to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/3eb4d4087d584c7f9627f94b7ee4146b 2023-08-07 08:53:24,120 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/4e64012a38374e2ca2434d2f7f29a7ec to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/4e64012a38374e2ca2434d2f7f29a7ec 2023-08-07 08:53:24,120 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/87b571fd8e7c468ab048a070d07e906f to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/cf2/87b571fd8e7c468ab048a070d07e906f 2023-08-07 08:53:24,120 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf2/0fc29b2b52b847cba7411c3b6e04901a to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/cf2/0fc29b2b52b847cba7411c3b6e04901a 2023-08-07 08:53:24,122 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/659eb35ea67c4594bf1b52bdb4a143f6 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/659eb35ea67c4594bf1b52bdb4a143f6 2023-08-07 08:53:24,124 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/932e1f93301447569e60673ba85254a8 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/932e1f93301447569e60673ba85254a8 2023-08-07 08:53:24,125 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/recovered.edits/90.seqid to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785/recovered.edits/90.seqid 2023-08-07 08:53:24,125 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/recovered.edits/90.seqid to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee/recovered.edits/90.seqid 2023-08-07 08:53:24,126 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d86003e731a97a463196e40fae27785 2023-08-07 08:53:24,126 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/46ac7a76f9146bf6f12230cccd2abdee 2023-08-07 08:53:24,126 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/d52c1e335593428cb634f8dc4ac90982 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/d52c1e335593428cb634f8dc4ac90982 2023-08-07 08:53:24,128 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/de3670c46dac41a0868e3ab198ad27fa to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf1/de3670c46dac41a0868e3ab198ad27fa 2023-08-07 08:53:24,132 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/22bf3f0bba424df3ab01f8dad19ab0ef to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/22bf3f0bba424df3ab01f8dad19ab0ef 2023-08-07 08:53:24,133 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/36228246fe7340d49afaf2d2784d8676 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/36228246fe7340d49afaf2d2784d8676 2023-08-07 08:53:24,135 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/39bb5ddbe47c4def9d6d0bbc388d94c3 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/39bb5ddbe47c4def9d6d0bbc388d94c3 2023-08-07 08:53:24,136 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/7825966e17d24694be7beb433a3cfa43 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/7825966e17d24694be7beb433a3cfa43 2023-08-07 08:53:24,138 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/a65ac280a1f34189883338289cc85639 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/a65ac280a1f34189883338289cc85639 2023-08-07 08:53:24,139 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/ba71af67ab9d4117bb3c565efc95c3b0 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/ba71af67ab9d4117bb3c565efc95c3b0 2023-08-07 08:53:24,141 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/ba9cb9a643e04291a2e4704d02d6dce1 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/cf2/ba9cb9a643e04291a2e4704d02d6dce1 2023-08-07 08:53:24,144 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/recovered.edits/85.seqid to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf/recovered.edits/85.seqid 2023-08-07 08:53:24,145 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionUnevenDaughter/769ddd8aa61f13dd3dc139c4fa3762cf 2023-08-07 08:53:24,145 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-08-07 08:53:24,148 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:24,174 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-08-07 08:53:24,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=24 2023-08-07 08:53:24,217 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-08-07 08:53:24,219 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:24,219 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-08-07 08:53:24,219 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691398404219"}]},"ts":"9223372036854775807"} 2023-08-07 08:53:24,219 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1691398401456.46ac7a76f9146bf6f12230cccd2abdee.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691398404219"}]},"ts":"9223372036854775807"} 2023-08-07 08:53:24,219 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691398404219"}]},"ts":"9223372036854775807"} 2023-08-07 08:53:24,223 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-08-07 08:53:24,223 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 769ddd8aa61f13dd3dc139c4fa3762cf, NAME => 'testSplitTableRegionUnevenDaughter,,1691398399312.769ddd8aa61f13dd3dc139c4fa3762cf.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 46ac7a76f9146bf6f12230cccd2abdee, NAME => 'testSplitTableRegionUnevenDaughter,,1691398401456.46ac7a76f9146bf6f12230cccd2abdee.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => 1d86003e731a97a463196e40fae27785, NAME => 'testSplitTableRegionUnevenDaughter,26,1691398401456.1d86003e731a97a463196e40fae27785.', STARTKEY => '26', ENDKEY => ''}] 2023-08-07 08:53:24,223 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-08-07 08:53:24,223 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1691398404223"}]},"ts":"9223372036854775807"} 2023-08-07 08:53:24,226 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-08-07 08:53:24,228 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:24,230 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 139 msec 2023-08-07 08:53:24,403 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=24 2023-08-07 08:53:24,403 INFO [Listener at localhost/33561] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-08-07 08:53:24,421 INFO [Listener at localhost/33561] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=391 (was 360) Potentially hanging thread: hconnection-0x36ae5be-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_63952265_17 at /127.0.0.1:56896 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2e450bc6-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_63952265_17 at /127.0.0.1:53284 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x36ae5be-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_63952265_17 at /127.0.0.1:33848 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_63952265_17 at /127.0.0.1:33698 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_63952265_17 at /127.0.0.1:53288 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2e450bc6-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:46525-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2e450bc6-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_63952265_17 at /127.0.0.1:43776 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2e450bc6-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_63952265_17 at /127.0.0.1:53310 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_63952265_17 at /127.0.0.1:56920 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_63952265_17 at /127.0.0.1:49878 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_63952265_17 at /127.0.0.1:53306 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_63952265_17 at /127.0.0.1:50146 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_63952265_17 at /127.0.0.1:53262 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x36ae5be-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x36ae5be-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_63952265_17 at /127.0.0.1:56914 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_63952265_17 at /127.0.0.1:56910 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_63952265_17 at /127.0.0.1:49910 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x36ae5be-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_63952265_17 at /127.0.0.1:56658 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2e450bc6-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x36ae5be-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=835 (was 700) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=374 (was 336) - SystemLoadAverage LEAK? -, ProcessCount=174 (was 174), AvailableMemoryMB=2015 (was 2083) 2023-08-07 08:53:24,436 INFO [Listener at localhost/33561] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=391, OpenFileDescriptor=835, MaxFileDescriptor=60000, SystemLoadAverage=374, ProcessCount=174, AvailableMemoryMB=2014 2023-08-07 08:53:24,436 WARN [Listener at localhost/33561] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-07 08:53:24,443 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-08-07 08:53:24,444 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-08-07 08:53:24,446 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:24,448 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-08-07 08:53:24,449 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-07 08:53:24,451 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:24,452 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd empty. 2023-08-07 08:53:24,452 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:24,452 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-08-07 08:53:24,465 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-08-07 08:53:24,466 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7675): creating {ENCODED => 7092830e4beb39626d41e32e9a87a8fd, NAME => 'testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp 2023-08-07 08:53:24,478 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:53:24,478 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1604): Closing 7092830e4beb39626d41e32e9a87a8fd, disabling compactions & flushes 2023-08-07 08:53:24,479 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd. 2023-08-07 08:53:24,479 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd. 2023-08-07 08:53:24,479 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd. after waiting 0 ms 2023-08-07 08:53:24,479 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd. 2023-08-07 08:53:24,479 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd. 2023-08-07 08:53:24,479 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1558): Region close journal for 7092830e4beb39626d41e32e9a87a8fd: 2023-08-07 08:53:24,481 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-08-07 08:53:24,482 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1691398404482"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398404482"}]},"ts":"1691398404482"} 2023-08-07 08:53:24,484 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-07 08:53:24,485 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-07 08:53:24,485 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398404485"}]},"ts":"1691398404485"} 2023-08-07 08:53:24,487 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-08-07 08:53:24,490 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-07 08:53:24,491 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-07 08:53:24,491 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-07 08:53:24,491 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-07 08:53:24,491 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-07 08:53:24,492 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd, ASSIGN}] 2023-08-07 08:53:24,494 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd, ASSIGN 2023-08-07 08:53:24,495 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46525,1691398396377; forceNewPlan=false, retain=false 2023-08-07 08:53:24,645 INFO [jenkins-hbase4:33357] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-07 08:53:24,646 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=7092830e4beb39626d41e32e9a87a8fd, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:24,646 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1691398404646"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398404646"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398404646"}]},"ts":"1691398404646"} 2023-08-07 08:53:24,648 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure 7092830e4beb39626d41e32e9a87a8fd, server=jenkins-hbase4.apache.org,46525,1691398396377}] 2023-08-07 08:53:24,805 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd. 2023-08-07 08:53:24,805 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 7092830e4beb39626d41e32e9a87a8fd, NAME => 'testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd.', STARTKEY => '', ENDKEY => ''} 2023-08-07 08:53:24,805 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-07 08:53:24,806 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:24,806 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:53:24,806 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:24,806 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:24,807 INFO [StoreOpener-7092830e4beb39626d41e32e9a87a8fd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:24,809 DEBUG [StoreOpener-7092830e4beb39626d41e32e9a87a8fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1 2023-08-07 08:53:24,809 DEBUG [StoreOpener-7092830e4beb39626d41e32e9a87a8fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1 2023-08-07 08:53:24,809 INFO [StoreOpener-7092830e4beb39626d41e32e9a87a8fd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7092830e4beb39626d41e32e9a87a8fd columnFamilyName cf1 2023-08-07 08:53:24,810 INFO [StoreOpener-7092830e4beb39626d41e32e9a87a8fd-1] regionserver.HStore(310): Store=7092830e4beb39626d41e32e9a87a8fd/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:24,810 INFO [StoreOpener-7092830e4beb39626d41e32e9a87a8fd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:24,812 DEBUG [StoreOpener-7092830e4beb39626d41e32e9a87a8fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2 2023-08-07 08:53:24,812 DEBUG [StoreOpener-7092830e4beb39626d41e32e9a87a8fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2 2023-08-07 08:53:24,814 INFO [StoreOpener-7092830e4beb39626d41e32e9a87a8fd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7092830e4beb39626d41e32e9a87a8fd columnFamilyName cf2 2023-08-07 08:53:24,815 INFO [StoreOpener-7092830e4beb39626d41e32e9a87a8fd-1] regionserver.HStore(310): Store=7092830e4beb39626d41e32e9a87a8fd/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:24,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:24,817 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:24,819 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-07 08:53:24,820 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:24,822 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 08:53:24,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 7092830e4beb39626d41e32e9a87a8fd; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69350097, jitterRate=0.033396974205970764}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-07 08:53:24,823 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 7092830e4beb39626d41e32e9a87a8fd: 2023-08-07 08:53:24,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd., pid=27, masterSystemTime=1691398404800 2023-08-07 08:53:24,826 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd. 2023-08-07 08:53:24,826 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd. 2023-08-07 08:53:24,827 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=7092830e4beb39626d41e32e9a87a8fd, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:24,827 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1691398404827"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691398404827"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691398404827"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398404827"}]},"ts":"1691398404827"} 2023-08-07 08:53:24,831 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=26 2023-08-07 08:53:24,831 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure 7092830e4beb39626d41e32e9a87a8fd, server=jenkins-hbase4.apache.org,46525,1691398396377 in 181 msec 2023-08-07 08:53:24,834 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=25 2023-08-07 08:53:24,834 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd, ASSIGN in 339 msec 2023-08-07 08:53:24,835 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-07 08:53:24,835 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398404835"}]},"ts":"1691398404835"} 2023-08-07 08:53:24,837 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-08-07 08:53:24,839 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-08-07 08:53:24,841 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 394 msec 2023-08-07 08:53:24,959 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:53:24,962 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-08-07 08:53:24,962 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-08-07 08:53:24,962 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:24,963 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-08-07 08:53:24,963 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-08-07 08:53:24,963 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:24,964 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:24,966 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:24,966 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:24,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:24,966 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:24,966 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:24,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:24,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:24,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:24,966 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:24,966 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:53:24,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:24,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:24,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:24,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:24,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:24,967 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:24,967 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:24,967 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:24,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:24,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:24,968 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-07 08:53:24,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:24,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:24,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:24,968 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-07 08:53:24,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:24,968 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-07 08:53:24,968 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-07 08:53:24,968 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-07 08:53:24,968 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:24,968 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-07 08:53:24,969 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-07 08:53:24,971 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-07 08:53:24,968 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-07 08:53:24,971 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:24,971 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-07 08:53:24,972 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:24,971 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-07 08:53:24,972 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-07 08:53:24,972 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-07 08:53:24,973 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-07 08:53:24,973 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-07 08:53:24,973 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-07 08:53:24,973 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:53:24,973 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:53:24,973 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd. 2023-08-07 08:53:24,973 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd. started... 2023-08-07 08:53:24,974 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7092830e4beb39626d41e32e9a87a8fd 2/2 column families, dataSize=120 B heapSize=928 B 2023-08-07 08:53:24,975 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:24,976 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:24,976 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:24,976 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:24,976 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:24,976 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:24,986 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/.tmp/cf1/ce7bda9f136841ae8c7db5220240286e 2023-08-07 08:53:25,002 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/.tmp/cf2/04734e257fcf493c8d959eba75318ca9 2023-08-07 08:53:25,008 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/.tmp/cf1/ce7bda9f136841ae8c7db5220240286e as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/ce7bda9f136841ae8c7db5220240286e 2023-08-07 08:53:25,012 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/ce7bda9f136841ae8c7db5220240286e, entries=2, sequenceid=6, filesize=4.8 K 2023-08-07 08:53:25,013 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/.tmp/cf2/04734e257fcf493c8d959eba75318ca9 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/04734e257fcf493c8d959eba75318ca9 2023-08-07 08:53:25,019 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/04734e257fcf493c8d959eba75318ca9, entries=2, sequenceid=6, filesize=4.8 K 2023-08-07 08:53:25,020 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 7092830e4beb39626d41e32e9a87a8fd in 46ms, sequenceid=6, compaction requested=false 2023-08-07 08:53:25,020 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-08-07 08:53:25,021 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7092830e4beb39626d41e32e9a87a8fd: 2023-08-07 08:53:25,021 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd. 2023-08-07 08:53:25,021 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:53:25,021 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:53:25,021 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,021 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-07 08:53:25,021 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-07 08:53:25,023 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,023 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:25,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:25,023 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,023 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:25,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:25,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:25,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-07 08:53:25,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:25,025 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-08-07 08:53:25,025 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7639dddf[Count = 0] remaining members to acquire global barrier 2023-08-07 08:53:25,025 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-08-07 08:53:25,025 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,028 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,028 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,028 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,028 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-07 08:53:25,028 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-07 08:53:25,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,028 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:53:25,028 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-07 08:53:25,028 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-07 08:53:25,028 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:53:25,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,028 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,028 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:53:25,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,029 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-07 08:53:25,029 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-07 08:53:25,029 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:53:25,030 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,030 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-07 08:53:25,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:25,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:25,030 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:25,030 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-07 08:53:25,031 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-07 08:53:25,031 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:25,031 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-07 08:53:25,031 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-07 08:53:25,032 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:25,032 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-07 08:53:25,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:25,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:25,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-07 08:53:25,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:25,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-07 08:53:25,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,46525,1691398396377': 2023-08-07 08:53:25,036 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-08-07 08:53:25,036 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-08-07 08:53:25,036 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:53:25,036 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:53:25,036 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-08-07 08:53:25,036 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:53:25,038 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,038 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,038 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:25,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,038 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:25,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,038 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:25,038 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:25,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:25,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:25,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:25,038 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:25,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:25,038 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:25,039 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:25,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:25,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-07 08:53:25,039 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:25,040 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-07 08:53:25,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:25,042 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-07 08:53:25,042 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,042 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,047 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,047 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:25,047 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:25,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:25,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:25,047 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:25,047 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,047 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:25,048 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,047 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:25,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:25,047 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:25,047 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-08-07 08:53:25,048 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:53:25,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:25,048 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,048 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:25,048 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:25,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:25,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:25,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:25,049 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-08-07 08:53:25,049 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:53:25,049 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:25,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:25,049 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,049 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,049 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,049 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,049 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:25,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:25,149 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:53:25,151 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:53:25,170 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:53:25,172 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-08-07 08:53:25,172 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-08-07 08:53:25,172 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-08-07 08:53:25,172 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:25,173 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-08-07 08:53:25,173 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-08-07 08:53:25,173 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,173 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,174 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,174 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:25,174 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:25,174 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:25,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:25,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:25,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:25,175 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,175 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:53:25,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:25,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:25,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:25,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,175 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,176 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,176 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:25,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:25,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,176 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-07 08:53:25,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:25,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,176 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-07 08:53:25,176 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-07 08:53:25,176 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-07 08:53:25,176 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-07 08:53:25,177 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-07 08:53:25,177 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-07 08:53:25,177 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-07 08:53:25,177 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-07 08:53:25,177 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-07 08:53:25,177 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:25,177 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:25,177 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-07 08:53:25,178 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-07 08:53:25,178 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-07 08:53:25,178 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:25,177 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-07 08:53:25,178 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-07 08:53:25,178 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-07 08:53:25,179 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:53:25,179 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:53:25,179 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-07 08:53:25,179 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd. 2023-08-07 08:53:25,179 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-07 08:53:25,179 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-07 08:53:25,179 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd. started... 2023-08-07 08:53:25,180 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7092830e4beb39626d41e32e9a87a8fd 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:53:25,181 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,182 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,182 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,182 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:25,182 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,182 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:25,194 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/.tmp/cf1/c41d6a566f664c6fa858730f8a1d5f30 2023-08-07 08:53:25,212 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/.tmp/cf2/a716ca57273f49e09d9c64e312525c9b 2023-08-07 08:53:25,219 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/.tmp/cf1/c41d6a566f664c6fa858730f8a1d5f30 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/c41d6a566f664c6fa858730f8a1d5f30 2023-08-07 08:53:25,228 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/c41d6a566f664c6fa858730f8a1d5f30, entries=10, sequenceid=19, filesize=5.0 K 2023-08-07 08:53:25,229 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/.tmp/cf2/a716ca57273f49e09d9c64e312525c9b as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/a716ca57273f49e09d9c64e312525c9b 2023-08-07 08:53:25,235 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/a716ca57273f49e09d9c64e312525c9b, entries=10, sequenceid=19, filesize=5.0 K 2023-08-07 08:53:25,236 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7092830e4beb39626d41e32e9a87a8fd in 56ms, sequenceid=19, compaction requested=false 2023-08-07 08:53:25,236 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7092830e4beb39626d41e32e9a87a8fd: 2023-08-07 08:53:25,236 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd. 2023-08-07 08:53:25,236 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:53:25,237 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:53:25,237 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,237 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-07 08:53:25,237 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-07 08:53:25,239 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,239 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:25,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:25,240 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,240 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:25,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:25,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:25,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-07 08:53:25,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:25,241 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-08-07 08:53:25,241 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6903482d[Count = 0] remaining members to acquire global barrier 2023-08-07 08:53:25,241 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-08-07 08:53:25,242 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,243 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,243 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,243 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,243 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-07 08:53:25,243 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,243 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:53:25,243 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-07 08:53:25,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,243 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-07 08:53:25,243 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-07 08:53:25,243 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:53:25,243 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:53:25,243 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-07 08:53:25,244 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-07 08:53:25,244 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:53:25,247 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-07 08:53:25,248 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,247 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-07 08:53:25,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:25,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:25,248 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:25,248 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:25,248 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-07 08:53:25,249 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-07 08:53:25,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:25,254 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-07 08:53:25,254 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:25,254 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-07 08:53:25,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:25,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-07 08:53:25,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:25,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-07 08:53:25,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,46525,1691398396377': 2023-08-07 08:53:25,259 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-08-07 08:53:25,259 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-08-07 08:53:25,259 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:53:25,259 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:53:25,259 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-08-07 08:53:25,259 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:53:25,261 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,261 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:25,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:25,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,261 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:25,261 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,261 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:25,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,261 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:25,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,261 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:25,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:25,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:25,262 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:25,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:25,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:25,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:25,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-07 08:53:25,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,263 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:25,263 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-07 08:53:25,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:25,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-07 08:53:25,266 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,266 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,267 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,270 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:25,270 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:25,270 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:25,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:25,270 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,270 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:25,270 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-08-07 08:53:25,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:25,270 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,270 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:25,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:25,270 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:25,271 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:25,270 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:53:25,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:25,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:25,271 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-08-07 08:53:25,271 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:53:25,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:25,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:25,270 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:25,271 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:25,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:25,272 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:25,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:25,272 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,272 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,272 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,272 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,272 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,272 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,340 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-07 08:53:25,341 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-08-07 08:53:25,342 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-07 08:53:25,342 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-08-07 08:53:25,372 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:53:25,373 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:53:25,391 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:53:25,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-08-07 08:53:25,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-08-07 08:53:25,393 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-08-07 08:53:25,393 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:25,394 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-08-07 08:53:25,394 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-08-07 08:53:25,395 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,395 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,396 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:25,396 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:25,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:25,396 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:25,396 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:25,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:25,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:25,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:25,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:25,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,397 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,397 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:53:25,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,397 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,398 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:25,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,398 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,398 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-07 08:53:25,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:25,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:25,398 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-07 08:53:25,398 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-07 08:53:25,398 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-07 08:53:25,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,399 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-07 08:53:25,399 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:25,399 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-07 08:53:25,399 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-07 08:53:25,399 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-07 08:53:25,399 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-07 08:53:25,399 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-07 08:53:25,399 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:25,399 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-07 08:53:25,399 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-07 08:53:25,400 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-07 08:53:25,400 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:25,400 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-07 08:53:25,401 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-07 08:53:25,401 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-07 08:53:25,400 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-07 08:53:25,401 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-07 08:53:25,401 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-07 08:53:25,401 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:53:25,401 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:53:25,401 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd. 2023-08-07 08:53:25,401 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd. started... 2023-08-07 08:53:25,401 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7092830e4beb39626d41e32e9a87a8fd 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:53:25,403 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,404 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,404 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,404 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:25,404 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,404 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:25,413 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/.tmp/cf1/11698c7cdc7b42579f078d865c7b69ee 2023-08-07 08:53:25,436 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/.tmp/cf2/a3f4c89d437d46de8b8573023aea4b4a 2023-08-07 08:53:25,442 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/.tmp/cf1/11698c7cdc7b42579f078d865c7b69ee as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/11698c7cdc7b42579f078d865c7b69ee 2023-08-07 08:53:25,447 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/11698c7cdc7b42579f078d865c7b69ee, entries=10, sequenceid=32, filesize=5.0 K 2023-08-07 08:53:25,448 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/.tmp/cf2/a3f4c89d437d46de8b8573023aea4b4a as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/a3f4c89d437d46de8b8573023aea4b4a 2023-08-07 08:53:25,453 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/a3f4c89d437d46de8b8573023aea4b4a, entries=10, sequenceid=32, filesize=5.0 K 2023-08-07 08:53:25,454 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7092830e4beb39626d41e32e9a87a8fd in 53ms, sequenceid=32, compaction requested=true 2023-08-07 08:53:25,454 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7092830e4beb39626d41e32e9a87a8fd: 2023-08-07 08:53:25,454 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd. 2023-08-07 08:53:25,454 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:53:25,454 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:53:25,454 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,454 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-07 08:53:25,454 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-07 08:53:25,456 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,456 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:25,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:25,456 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:25,456 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:25,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:25,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-07 08:53:25,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:25,458 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-08-07 08:53:25,458 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@62193a35[Count = 0] remaining members to acquire global barrier 2023-08-07 08:53:25,458 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-08-07 08:53:25,459 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,460 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,460 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,460 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,460 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-07 08:53:25,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,460 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,460 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:53:25,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,460 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-07 08:53:25,460 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-07 08:53:25,460 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-07 08:53:25,460 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-07 08:53:25,460 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:53:25,460 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-07 08:53:25,460 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:53:25,461 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:53:25,463 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,463 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-07 08:53:25,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:25,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:25,463 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:25,463 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-07 08:53:25,464 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-07 08:53:25,464 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-07 08:53:25,464 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:25,464 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:25,464 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-07 08:53:25,464 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-07 08:53:25,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:25,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:25,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-07 08:53:25,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:25,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-07 08:53:25,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,46525,1691398396377': 2023-08-07 08:53:25,467 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-08-07 08:53:25,467 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-08-07 08:53:25,467 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:53:25,467 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:53:25,467 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-08-07 08:53:25,467 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:53:25,469 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,469 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:25,470 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,470 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,470 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:25,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:25,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,470 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:25,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,470 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:25,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:25,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:25,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:25,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:25,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:25,471 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:25,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:25,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,471 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-07 08:53:25,471 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:25,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-07 08:53:25,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,473 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:25,474 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-07 08:53:25,474 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,477 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:25,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:25,478 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:25,478 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:25,478 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:25,477 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,478 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-08-07 08:53:25,478 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:53:25,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:25,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:25,478 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:25,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:25,478 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:25,478 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,479 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,479 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,479 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,479 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,479 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,479 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,479 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,479 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,479 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:25,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:25,478 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:25,482 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-08-07 08:53:25,483 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:53:25,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:25,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:25,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:25,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:25,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:25,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:25,583 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:53:25,584 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:53:25,604 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:53:25,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-08-07 08:53:25,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-08-07 08:53:25,606 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-08-07 08:53:25,606 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:25,607 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-08-07 08:53:25,607 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-08-07 08:53:25,607 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,608 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,610 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:25,610 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:25,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:25,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:25,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:25,610 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,610 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:25,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:25,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:25,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:25,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,611 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,611 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:53:25,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,611 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,611 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,612 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:25,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:25,612 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-07 08:53:25,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,612 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-07 08:53:25,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:25,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,612 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-07 08:53:25,612 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-07 08:53:25,612 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-07 08:53:25,612 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-07 08:53:25,612 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-07 08:53:25,612 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-07 08:53:25,613 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:25,613 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-07 08:53:25,613 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-07 08:53:25,614 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-07 08:53:25,613 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:25,613 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-07 08:53:25,614 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-07 08:53:25,614 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-07 08:53:25,614 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-07 08:53:25,614 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-07 08:53:25,614 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-07 08:53:25,615 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-07 08:53:25,615 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:25,617 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-07 08:53:25,617 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:53:25,617 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:53:25,617 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd. 2023-08-07 08:53:25,617 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd. started... 2023-08-07 08:53:25,617 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7092830e4beb39626d41e32e9a87a8fd 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:53:25,621 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,621 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,621 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,621 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:25,621 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,621 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:25,631 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/.tmp/cf1/f1f56fa60c5c4350be33a4b770642c31 2023-08-07 08:53:25,649 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/.tmp/cf2/dab875aff32b43609cfd694e172281e6 2023-08-07 08:53:25,655 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/.tmp/cf1/f1f56fa60c5c4350be33a4b770642c31 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/f1f56fa60c5c4350be33a4b770642c31 2023-08-07 08:53:25,660 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/f1f56fa60c5c4350be33a4b770642c31, entries=10, sequenceid=45, filesize=5.0 K 2023-08-07 08:53:25,661 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/.tmp/cf2/dab875aff32b43609cfd694e172281e6 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dab875aff32b43609cfd694e172281e6 2023-08-07 08:53:25,666 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dab875aff32b43609cfd694e172281e6, entries=10, sequenceid=45, filesize=5.0 K 2023-08-07 08:53:25,667 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7092830e4beb39626d41e32e9a87a8fd in 50ms, sequenceid=45, compaction requested=true 2023-08-07 08:53:25,667 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7092830e4beb39626d41e32e9a87a8fd: 2023-08-07 08:53:25,668 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd. 2023-08-07 08:53:25,668 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:53:25,668 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:53:25,668 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,668 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-07 08:53:25,668 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-07 08:53:25,669 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,669 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:25,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:25,670 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,670 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:25,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:25,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:25,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-07 08:53:25,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:25,672 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-08-07 08:53:25,672 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@64aed07c[Count = 0] remaining members to acquire global barrier 2023-08-07 08:53:25,672 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-08-07 08:53:25,672 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,674 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,674 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,674 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,674 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-07 08:53:25,674 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-07 08:53:25,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,674 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,674 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:53:25,674 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-07 08:53:25,674 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:53:25,674 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-07 08:53:25,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,674 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:53:25,674 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-07 08:53:25,675 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-07 08:53:25,675 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:53:25,677 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-07 08:53:25,677 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:25,677 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-07 08:53:25,677 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-07 08:53:25,677 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:25,677 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-07 08:53:25,678 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,678 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-07 08:53:25,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:25,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:25,678 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:25,678 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-07 08:53:25,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:25,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:25,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-07 08:53:25,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:25,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-07 08:53:25,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,46525,1691398396377': 2023-08-07 08:53:25,682 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-08-07 08:53:25,682 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-08-07 08:53:25,682 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:53:25,682 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:53:25,682 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-08-07 08:53:25,682 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:53:25,684 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,684 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,684 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,684 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:25,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:25,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:25,684 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,684 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:25,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:25,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,685 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:25,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:25,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:25,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,685 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:25,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:25,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-07 08:53:25,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:25,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:25,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,686 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:25,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,687 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-07 08:53:25,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:25,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-07 08:53:25,688 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,689 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,689 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,693 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:25,693 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:25,693 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:25,693 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:25,693 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-08-07 08:53:25,693 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:53:25,693 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:25,693 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:25,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:25,693 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:25,693 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,693 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:25,693 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:25,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:25,693 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:25,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:25,694 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-08-07 08:53:25,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:25,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:25,694 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:25,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:25,694 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:53:25,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:25,694 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:25,695 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,695 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,695 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,695 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,695 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,794 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:53:25,796 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:53:25,818 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:53:25,820 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-08-07 08:53:25,821 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-08-07 08:53:25,821 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-08-07 08:53:25,821 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:25,821 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-08-07 08:53:25,821 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-08-07 08:53:25,821 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,821 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,823 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,823 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:25,823 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:25,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:25,823 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:25,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:25,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:25,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:25,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:25,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:25,823 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,823 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:53:25,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,824 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,824 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,824 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:25,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:25,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,824 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-07 08:53:25,824 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-07 08:53:25,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:25,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,825 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-07 08:53:25,825 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-07 08:53:25,825 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-07 08:53:25,825 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-07 08:53:25,825 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:25,825 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-07 08:53:25,825 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-07 08:53:25,825 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-07 08:53:25,825 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-07 08:53:25,825 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:25,825 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-07 08:53:25,825 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-07 08:53:25,825 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-07 08:53:25,826 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-07 08:53:25,826 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-07 08:53:25,827 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:53:25,827 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,827 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-07 08:53:25,827 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:25,827 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd. 2023-08-07 08:53:25,827 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd. started... 2023-08-07 08:53:25,827 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:53:25,827 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,827 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:25,827 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-07 08:53:25,827 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7092830e4beb39626d41e32e9a87a8fd 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:53:25,827 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-07 08:53:25,827 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-07 08:53:25,829 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,829 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,829 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:25,846 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/.tmp/cf1/bd6bd562e3df4563b70a5aad06a1c77c 2023-08-07 08:53:25,862 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/.tmp/cf2/dbdcd43d7530461db851264cc92dc513 2023-08-07 08:53:25,868 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/.tmp/cf1/bd6bd562e3df4563b70a5aad06a1c77c as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/bd6bd562e3df4563b70a5aad06a1c77c 2023-08-07 08:53:25,873 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/bd6bd562e3df4563b70a5aad06a1c77c, entries=10, sequenceid=58, filesize=5.0 K 2023-08-07 08:53:25,874 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/.tmp/cf2/dbdcd43d7530461db851264cc92dc513 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dbdcd43d7530461db851264cc92dc513 2023-08-07 08:53:25,880 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dbdcd43d7530461db851264cc92dc513, entries=10, sequenceid=58, filesize=5.0 K 2023-08-07 08:53:25,881 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7092830e4beb39626d41e32e9a87a8fd in 54ms, sequenceid=58, compaction requested=true 2023-08-07 08:53:25,881 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7092830e4beb39626d41e32e9a87a8fd: 2023-08-07 08:53:25,881 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd. 2023-08-07 08:53:25,881 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:53:25,881 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:53:25,881 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,881 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-07 08:53:25,881 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-07 08:53:25,884 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,884 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:25,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:25,884 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,884 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:25,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:25,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:25,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-07 08:53:25,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:25,886 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-08-07 08:53:25,886 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5cc12e13[Count = 0] remaining members to acquire global barrier 2023-08-07 08:53:25,886 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-08-07 08:53:25,887 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,888 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,888 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,888 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,888 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-07 08:53:25,888 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-07 08:53:25,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,888 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,888 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:53:25,888 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:53:25,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,889 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-07 08:53:25,889 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-07 08:53:25,889 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:53:25,889 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-07 08:53:25,889 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-07 08:53:25,889 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:53:25,890 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-07 08:53:25,890 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:25,890 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-07 08:53:25,891 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,891 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-07 08:53:25,891 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:25,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:25,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:25,891 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-07 08:53:25,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:25,892 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-07 08:53:25,892 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:25,892 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-07 08:53:25,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:25,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-07 08:53:25,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:25,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-07 08:53:25,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,46525,1691398396377': 2023-08-07 08:53:25,895 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-08-07 08:53:25,895 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-08-07 08:53:25,895 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:53:25,895 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:53:25,896 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-08-07 08:53:25,896 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:53:25,897 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,897 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:25,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:25,897 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:25,897 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,897 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,897 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:25,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,898 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:25,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,897 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:25,898 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,898 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:25,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:25,898 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:25,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:25,898 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:25,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:25,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-07 08:53:25,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,899 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:25,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-07 08:53:25,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,901 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:25,901 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-07 08:53:25,901 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,906 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,906 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:25,906 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:25,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:25,906 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:25,906 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-08-07 08:53:25,906 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:53:25,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:25,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:25,906 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:25,906 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,906 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:25,907 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,907 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:25,907 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,906 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:25,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:25,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:25,907 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-08-07 08:53:25,907 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:53:25,907 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:25,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:25,907 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:25,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:25,907 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:25,908 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:25,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:25,908 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:25,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:25,908 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:25,908 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:25,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:25,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:26,007 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:53:26,009 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:53:26,030 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:53:26,032 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-08-07 08:53:26,032 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-08-07 08:53:26,032 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-08-07 08:53:26,032 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:26,032 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-08-07 08:53:26,032 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-08-07 08:53:26,033 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:26,033 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:26,034 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:26,034 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:26,034 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:26,034 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:26,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:26,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:26,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:26,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:26,035 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:26,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:26,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:26,035 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:53:26,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:26,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:26,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:26,035 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:26,036 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:26,036 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:26,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:26,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:26,036 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-07 08:53:26,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:26,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:26,036 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-07 08:53:26,036 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-07 08:53:26,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:26,036 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-07 08:53:26,036 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-07 08:53:26,037 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-07 08:53:26,037 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-07 08:53:26,037 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-07 08:53:26,037 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:26,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:26,037 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-07 08:53:26,037 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:26,037 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-07 08:53:26,037 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-07 08:53:26,037 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-07 08:53:26,037 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-07 08:53:26,037 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:53:26,037 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:53:26,037 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd. 2023-08-07 08:53:26,037 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-07 08:53:26,037 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd. started... 2023-08-07 08:53:26,037 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-07 08:53:26,037 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-07 08:53:26,037 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7092830e4beb39626d41e32e9a87a8fd 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:53:26,037 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:26,037 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-07 08:53:26,038 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-07 08:53:26,038 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-07 08:53:26,038 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:26,041 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:26,041 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:26,041 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:26,041 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:26,042 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:26,055 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/.tmp/cf1/6a5e859afc8b45509790593454f7da3d 2023-08-07 08:53:26,074 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/.tmp/cf2/0a13b583104d4bf786ba47b94382f3df 2023-08-07 08:53:26,080 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/.tmp/cf1/6a5e859afc8b45509790593454f7da3d as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/6a5e859afc8b45509790593454f7da3d 2023-08-07 08:53:26,086 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/6a5e859afc8b45509790593454f7da3d, entries=10, sequenceid=71, filesize=5.0 K 2023-08-07 08:53:26,087 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/.tmp/cf2/0a13b583104d4bf786ba47b94382f3df as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/0a13b583104d4bf786ba47b94382f3df 2023-08-07 08:53:26,092 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/0a13b583104d4bf786ba47b94382f3df, entries=10, sequenceid=71, filesize=5.0 K 2023-08-07 08:53:26,093 INFO [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7092830e4beb39626d41e32e9a87a8fd in 56ms, sequenceid=71, compaction requested=true 2023-08-07 08:53:26,093 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7092830e4beb39626d41e32e9a87a8fd: 2023-08-07 08:53:26,093 DEBUG [rs(jenkins-hbase4.apache.org,46525,1691398396377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd. 2023-08-07 08:53:26,093 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:53:26,093 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:53:26,093 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:26,093 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-07 08:53:26,093 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-07 08:53:26,095 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:26,095 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:26,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:26,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:26,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:26,095 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:26,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:26,096 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:26,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:26,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-07 08:53:26,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:26,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:26,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:26,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:26,097 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-08-07 08:53:26,098 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2ff05786[Count = 0] remaining members to acquire global barrier 2023-08-07 08:53:26,098 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-08-07 08:53:26,098 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:26,099 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:26,099 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:26,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:26,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:26,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:26,099 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:26,099 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-07 08:53:26,099 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-07 08:53:26,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:26,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:26,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:26,099 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-07 08:53:26,099 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-07 08:53:26,099 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:26,099 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:53:26,099 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:53:26,099 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:53:26,099 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-07 08:53:26,100 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-07 08:53:26,100 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:53:26,101 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-07 08:53:26,101 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:26,101 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:26,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:26,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:26,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:26,101 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-07 08:53:26,102 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-07 08:53:26,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:26,102 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:26,102 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-07 08:53:26,102 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:26,102 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-07 08:53:26,102 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-07 08:53:26,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:26,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-07 08:53:26,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:26,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:26,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:26,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:26,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-07 08:53:26,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:26,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:26,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:26,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,46525,1691398396377': 2023-08-07 08:53:26,105 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-08-07 08:53:26,105 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-08-07 08:53:26,105 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:53:26,105 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:53:26,105 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-08-07 08:53:26,105 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:53:26,107 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:26,107 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:26,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:26,107 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:26,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:26,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:26,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:26,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:26,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:26,107 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:26,107 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:26,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:26,107 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:26,108 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:26,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:26,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:26,107 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:26,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:26,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:26,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:26,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:26,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:26,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:26,108 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:26,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:26,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-07 08:53:26,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:26,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:26,109 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:26,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:26,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:26,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-07 08:53:26,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:26,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:26,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:26,110 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:26,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:26,111 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:26,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-07 08:53:26,111 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:26,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:26,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:26,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:26,114 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:26,114 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:26,114 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:26,114 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:26,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:26,114 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:26,114 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:26,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:26,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:26,114 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:26,114 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-08-07 08:53:26,114 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:53:26,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:26,114 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:26,114 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:26,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:26,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:26,114 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:26,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:26,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:26,115 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-08-07 08:53:26,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:26,114 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:26,115 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:53:26,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:26,115 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-07 08:53:26,115 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:26,115 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:26,115 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:26,115 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:26,115 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-07 08:53:26,115 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-07 08:53:26,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:26,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:26,215 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:53:26,216 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:53:26,479 WARN [Listener at localhost/33561] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-08-07 08:53:26,480 DEBUG [Listener at localhost/33561] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:26,482 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(1029): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:26,483 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:26,483 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:26,483 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:26,484 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:53:26,484 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:26,484 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:53:26,485 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:53:26,732 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:26,734 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:53:26,734 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:53:26,734 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:53:26,736 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-08-07 08:53:26,737 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:53:26,738 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:53:26,738 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:53:26,738 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:53:26,739 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:53:26,741 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-08-07 08:53:26,743 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:53:26,743 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:53:26,744 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:53:26,744 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:53:26,744 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:26,745 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:53:26,745 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:26,746 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:26,746 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:26,747 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:26,747 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:53:26,747 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:26,747 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:26,747 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:26,747 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-08-07 08:53:26,748 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:53:26,749 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:53:26,751 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398436750 2023-08-07 08:53:26,752 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:53:26,752 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:26,755 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:26,756 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=7092830e4beb39626d41e32e9a87a8fd, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:26,756 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:53:26,756 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398466756 2023-08-07 08:53:26,756 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398526756 2023-08-07 08:53:26,756 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-08-07 08:53:26,756 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:53:26,759 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:53:26,759 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:26,759 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:53:26,759 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:53:27,006 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:27,006 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:53:27,007 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:53:27,007 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:53:27,007 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 08:53:27,007 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:53:27,007 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:53:27,007 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:53:27,007 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:53:27,008 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:53:27,009 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-08-07 08:53:27,010 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:53:27,011 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:53:27,011 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:53:27,011 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:53:27,011 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:27,012 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:53:27,012 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:27,012 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:27,013 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:27,013 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:27,013 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:53:27,013 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:27,013 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:27,013 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:27,014 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-08-07 08:53:27,014 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:53:27,015 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:53:27,015 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398437015 2023-08-07 08:53:27,017 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:27,017 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:53:27,019 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:27,020 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=7092830e4beb39626d41e32e9a87a8fd, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:27,020 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:53:27,020 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398467020 2023-08-07 08:53:27,020 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398527020 2023-08-07 08:53:27,020 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-07 08:53:27,020 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:53:27,022 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd, UNASSIGN}] 2023-08-07 08:53:27,022 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:53:27,022 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:27,022 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:53:27,022 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:53:27,270 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:27,270 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:53:27,270 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:53:27,271 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:53:27,271 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 08:53:27,271 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:53:27,271 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:53:27,271 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:53:27,271 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:53:27,272 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:53:27,273 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-08-07 08:53:27,274 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:53:27,274 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:53:27,274 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:53:27,275 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:53:27,275 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:27,276 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:53:27,276 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:27,276 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:27,276 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:27,277 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:27,277 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:53:27,277 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:27,277 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:27,277 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:27,277 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-08-07 08:53:27,278 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:53:27,278 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:53:27,279 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398437279 2023-08-07 08:53:27,279 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:53:27,280 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:27,283 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:27,283 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=7092830e4beb39626d41e32e9a87a8fd, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:27,283 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:53:27,283 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398467283 2023-08-07 08:53:27,283 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398527283 2023-08-07 08:53:27,283 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-08-07 08:53:27,283 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd, UNASSIGN}] 2023-08-07 08:53:27,284 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:53:27,286 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd, UNASSIGN 2023-08-07 08:53:27,286 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=7092830e4beb39626d41e32e9a87a8fd, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:27,287 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1691398407286"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398407286"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398407286"}]},"ts":"1691398407286"} 2023-08-07 08:53:27,288 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 7092830e4beb39626d41e32e9a87a8fd, server=jenkins-hbase4.apache.org,46525,1691398396377}] 2023-08-07 08:53:27,288 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:53:27,288 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd, UNASSIGN 2023-08-07 08:53:27,288 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:53:27,288 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:53:27,533 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:27,534 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:53:27,534 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:53:27,534 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:53:27,534 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 08:53:27,534 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:53:27,534 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:53:27,534 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:53:27,534 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:53:27,535 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:53:27,537 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-08-07 08:53:27,537 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-08-07 08:53:27,537 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:53:27,537 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:53:27,538 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:53:27,538 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:53:27,538 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:27,539 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:53:27,539 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:27,539 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:27,539 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:27,540 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:27,540 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd, UNASSIGN 2023-08-07 08:53:27,540 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:53:27,541 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:27,541 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:27,541 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:27,541 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:53:27,541 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd, UNASSIGN 2023-08-07 08:53:27,541 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-08-07 08:53:27,542 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:53:27,542 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:53:27,542 INFO [Listener at localhost/33561] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd to restore RIT 2023-08-07 08:53:27,543 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398437543 2023-08-07 08:53:27,544 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:53:27,544 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd, UNASSIGN 2023-08-07 08:53:27,546 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:27,546 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=7092830e4beb39626d41e32e9a87a8fd, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:27,546 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:53:27,546 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398467546 2023-08-07 08:53:27,547 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398527547 2023-08-07 08:53:27,547 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-07 08:53:27,547 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=7092830e4beb39626d41e32e9a87a8fd, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:27,547 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1691398407547"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398407547"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398407547"}]},"ts":"1691398407547"} 2023-08-07 08:53:27,549 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 7092830e4beb39626d41e32e9a87a8fd, server=jenkins-hbase4.apache.org,46525,1691398396377}] 2023-08-07 08:53:27,549 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:53:27,703 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:27,703 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 7092830e4beb39626d41e32e9a87a8fd, disabling compactions & flushes 2023-08-07 08:53:27,703 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd. 2023-08-07 08:53:27,703 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd. 2023-08-07 08:53:27,703 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd. after waiting 0 ms 2023-08-07 08:53:27,703 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd. 2023-08-07 08:53:27,703 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 7092830e4beb39626d41e32e9a87a8fd 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-08-07 08:53:27,714 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/.tmp/cf1/805e8b3c0672407a856b2e9957e86916 2023-08-07 08:53:27,737 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/.tmp/cf2/dba9d7b0dd5743838d8dd8c952a368a9 2023-08-07 08:53:27,743 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/.tmp/cf1/805e8b3c0672407a856b2e9957e86916 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/805e8b3c0672407a856b2e9957e86916 2023-08-07 08:53:27,748 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/805e8b3c0672407a856b2e9957e86916, entries=8, sequenceid=82, filesize=5.0 K 2023-08-07 08:53:27,749 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/.tmp/cf2/dba9d7b0dd5743838d8dd8c952a368a9 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dba9d7b0dd5743838d8dd8c952a368a9 2023-08-07 08:53:27,754 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dba9d7b0dd5743838d8dd8c952a368a9, entries=8, sequenceid=82, filesize=5.0 K 2023-08-07 08:53:27,755 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 7092830e4beb39626d41e32e9a87a8fd in 52ms, sequenceid=82, compaction requested=true 2023-08-07 08:53:27,781 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-08-07 08:53:27,781 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-07 08:53:27,782 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd. 2023-08-07 08:53:27,782 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 7092830e4beb39626d41e32e9a87a8fd: 2023-08-07 08:53:27,785 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:27,785 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=7092830e4beb39626d41e32e9a87a8fd, regionState=CLOSED 2023-08-07 08:53:27,785 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1691398407785"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398407785"}]},"ts":"1691398407785"} 2023-08-07 08:53:27,787 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:53:27,787 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 7092830e4beb39626d41e32e9a87a8fd, server=jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:27,787 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:53:27,787 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 7092830e4beb39626d41e32e9a87a8fd, server=jenkins-hbase4.apache.org,46525,1691398396377 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:53:27,797 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:27,797 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:53:27,797 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:53:27,797 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:53:27,798 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 08:53:27,798 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:53:27,798 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:53:27,798 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:53:27,798 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:53:27,799 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:53:27,800 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-08-07 08:53:27,800 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-08-07 08:53:27,800 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-08-07 08:53:27,800 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:53:27,801 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:53:27,801 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:53:27,801 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:53:27,801 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:27,802 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:53:27,802 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:27,802 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:27,802 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:27,803 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:27,803 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd, UNASSIGN 2023-08-07 08:53:27,803 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 7092830e4beb39626d41e32e9a87a8fd, server=jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:27,803 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:53:27,803 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:27,803 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:27,804 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:27,804 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:53:27,804 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd, UNASSIGN 2023-08-07 08:53:27,804 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 7092830e4beb39626d41e32e9a87a8fd, server=jenkins-hbase4.apache.org,46525,1691398396377 held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:53:27,804 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-08-07 08:53:27,804 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:53:27,805 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:53:27,805 INFO [Listener at localhost/33561] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd to restore RIT 2023-08-07 08:53:27,805 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398437805 2023-08-07 08:53:27,806 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 7092830e4beb39626d41e32e9a87a8fd, server=jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:27,806 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:53:27,808 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:27,809 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=7092830e4beb39626d41e32e9a87a8fd, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:27,809 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:53:27,809 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398467809 2023-08-07 08:53:27,809 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398527809 2023-08-07 08:53:27,809 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-08-07 08:53:27,809 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=7092830e4beb39626d41e32e9a87a8fd, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:27,809 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1691398407809"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398407809"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398407809"}]},"ts":"1691398407809"} 2023-08-07 08:53:27,811 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:53:27,814 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=29 2023-08-07 08:53:27,814 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7092830e4beb39626d41e32e9a87a8fd, server=jenkins-hbase4.apache.org,46525,1691398396377 in 262 msec 2023-08-07 08:53:27,815 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:53:27,815 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd, UNASSIGN 2023-08-07 08:53:27,815 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:53:27,815 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:53:28,059 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:28,059 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:53:28,059 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:53:28,059 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:53:28,060 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 08:53:28,060 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:53:28,060 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:53:28,060 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:53:28,060 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:53:28,061 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:53:28,062 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-08-07 08:53:28,063 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-08-07 08:53:28,063 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-07 08:53:28,063 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:53:28,064 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:53:28,064 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:53:28,064 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:53:28,065 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:28,065 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:53:28,066 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:28,066 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:28,066 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:28,067 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:28,067 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd, UNASSIGN 2023-08-07 08:53:28,067 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7092830e4beb39626d41e32e9a87a8fd, server=jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:28,067 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:53:28,068 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:28,068 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:28,068 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:28,068 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:53:28,068 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7092830e4beb39626d41e32e9a87a8fd, server=jenkins-hbase4.apache.org,46525,1691398396377 didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:53:28,068 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-08-07 08:53:28,069 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:53:28,070 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:53:28,070 INFO [Listener at localhost/33561] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd to restore RIT 2023-08-07 08:53:28,075 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398438075 2023-08-07 08:53:28,076 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd, UNASSIGN 2023-08-07 08:53:28,076 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:53:28,078 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:28,079 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=7092830e4beb39626d41e32e9a87a8fd, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:28,079 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:53:28,079 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398468079 2023-08-07 08:53:28,079 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398528079 2023-08-07 08:53:28,079 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-07 08:53:28,079 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd, UNASSIGN 2023-08-07 08:53:28,081 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:53:28,084 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-08-07 08:53:28,084 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd, UNASSIGN in 798 msec 2023-08-07 08:53:28,087 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:53:28,087 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:28,087 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:53:28,087 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:53:28,329 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:28,329 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:53:28,329 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:53:28,329 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:53:28,330 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 08:53:28,330 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:53:28,330 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:53:28,330 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:53:28,330 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:53:28,331 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:53:28,332 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-08-07 08:53:28,332 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-07 08:53:28,332 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-07 08:53:28,333 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:53:28,333 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:53:28,334 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:53:28,334 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:53:28,334 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:28,335 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:53:28,335 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:28,335 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:28,335 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:28,336 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:28,336 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd, UNASSIGN 2023-08-07 08:53:28,336 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7092830e4beb39626d41e32e9a87a8fd, server=jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:28,336 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:53:28,336 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:53:28,336 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7092830e4beb39626d41e32e9a87a8fd, server=jenkins-hbase4.apache.org,46525,1691398396377 didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:53:28,336 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-08-07 08:53:28,337 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:53:28,338 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:53:28,338 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398438338 2023-08-07 08:53:28,339 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:53:28,340 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:28,342 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:28,343 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=7092830e4beb39626d41e32e9a87a8fd, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:28,343 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:53:28,343 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398468343 2023-08-07 08:53:28,343 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398528343 2023-08-07 08:53:28,343 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-08-07 08:53:28,343 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:28,343 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:28,343 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:28,349 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:53:28,359 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=7092830e4beb39626d41e32e9a87a8fd, threads=14 2023-08-07 08:53:28,360 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/04734e257fcf493c8d959eba75318ca9 for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,360 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/0a13b583104d4bf786ba47b94382f3df for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,360 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/a3f4c89d437d46de8b8573023aea4b4a for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,360 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/a716ca57273f49e09d9c64e312525c9b for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,361 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dab875aff32b43609cfd694e172281e6 for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,361 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dba9d7b0dd5743838d8dd8c952a368a9 for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,361 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dbdcd43d7530461db851264cc92dc513 for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,362 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/11698c7cdc7b42579f078d865c7b69ee for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,366 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/6a5e859afc8b45509790593454f7da3d for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,367 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/805e8b3c0672407a856b2e9957e86916 for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,367 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/bd6bd562e3df4563b70a5aad06a1c77c for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,367 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/c41d6a566f664c6fa858730f8a1d5f30 for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,367 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/ce7bda9f136841ae8c7db5220240286e for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,369 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/f1f56fa60c5c4350be33a4b770642c31 for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,536 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/a716ca57273f49e09d9c64e312525c9b for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,543 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/805e8b3c0672407a856b2e9957e86916 for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,546 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/0a13b583104d4bf786ba47b94382f3df for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,563 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dab875aff32b43609cfd694e172281e6 for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,579 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/04734e257fcf493c8d959eba75318ca9 for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,579 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/6a5e859afc8b45509790593454f7da3d for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,583 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/11698c7cdc7b42579f078d865c7b69ee for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,588 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/bd6bd562e3df4563b70a5aad06a1c77c for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,598 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dbdcd43d7530461db851264cc92dc513 for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,603 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/ce7bda9f136841ae8c7db5220240286e for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,603 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dba9d7b0dd5743838d8dd8c952a368a9 for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,606 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/a3f4c89d437d46de8b8573023aea4b4a for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,611 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/f1f56fa60c5c4350be33a4b770642c31 for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,613 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/c41d6a566f664c6fa858730f8a1d5f30 for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,613 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 7092830e4beb39626d41e32e9a87a8fd Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-08-07 08:53:28,652 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:53:28,652 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:28,652 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:53:28,652 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:53:28,843 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:28,843 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:53:28,844 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:53:28,844 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:53:28,844 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 08:53:28,844 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:53:28,844 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:53:28,844 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:53:28,844 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:53:28,845 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:53:28,847 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-08-07 08:53:28,847 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-07 08:53:28,847 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-07 08:53:28,848 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:53:28,848 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:53:28,849 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:53:28,849 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:53:28,849 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:28,850 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:53:28,850 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:28,850 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:28,851 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:28,852 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:28,852 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd, UNASSIGN 2023-08-07 08:53:28,852 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7092830e4beb39626d41e32e9a87a8fd, server=jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:28,852 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:53:28,853 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:28,853 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:28,853 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:28,853 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:53:28,853 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7092830e4beb39626d41e32e9a87a8fd, server=jenkins-hbase4.apache.org,46525,1691398396377 didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:53:28,853 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-08-07 08:53:28,855 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:53:28,855 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:53:28,856 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398438856 2023-08-07 08:53:28,858 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:53:28,858 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:28,860 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:28,861 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=7092830e4beb39626d41e32e9a87a8fd, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:28,861 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:53:28,861 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398468861 2023-08-07 08:53:28,861 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398528861 2023-08-07 08:53:28,861 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-07 08:53:28,872 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=7092830e4beb39626d41e32e9a87a8fd, threads=14 2023-08-07 08:53:28,873 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/04734e257fcf493c8d959eba75318ca9 for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,873 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/0a13b583104d4bf786ba47b94382f3df for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,873 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/a3f4c89d437d46de8b8573023aea4b4a for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,874 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/a716ca57273f49e09d9c64e312525c9b for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,874 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dab875aff32b43609cfd694e172281e6 for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,874 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dba9d7b0dd5743838d8dd8c952a368a9 for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,874 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dbdcd43d7530461db851264cc92dc513 for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,875 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/11698c7cdc7b42579f078d865c7b69ee for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,875 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/6a5e859afc8b45509790593454f7da3d for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,875 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/805e8b3c0672407a856b2e9957e86916 for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,876 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/bd6bd562e3df4563b70a5aad06a1c77c for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,876 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/c41d6a566f664c6fa858730f8a1d5f30 for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,876 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/ce7bda9f136841ae8c7db5220240286e for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:28,876 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/f1f56fa60c5c4350be33a4b770642c31 for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:29,046 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/04734e257fcf493c8d959eba75318ca9 for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:29,058 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/0a13b583104d4bf786ba47b94382f3df for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:29,058 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dba9d7b0dd5743838d8dd8c952a368a9 for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:29,073 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/805e8b3c0672407a856b2e9957e86916 for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:29,103 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/6a5e859afc8b45509790593454f7da3d for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:29,108 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/bd6bd562e3df4563b70a5aad06a1c77c for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:29,110 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dab875aff32b43609cfd694e172281e6 for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:29,118 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/11698c7cdc7b42579f078d865c7b69ee for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:29,131 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/ce7bda9f136841ae8c7db5220240286e for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:29,137 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/a716ca57273f49e09d9c64e312525c9b for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:29,137 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/c41d6a566f664c6fa858730f8a1d5f30 for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:29,139 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/a3f4c89d437d46de8b8573023aea4b4a for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:29,139 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dbdcd43d7530461db851264cc92dc513 for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:29,142 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/f1f56fa60c5c4350be33a4b770642c31 for region: 7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:29,142 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 7092830e4beb39626d41e32e9a87a8fd Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-08-07 08:53:29,207 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-08-07 08:53:29,587 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:53:29,593 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-08-07 08:53:29,595 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-08-07 08:53:29,595 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:53:29,595 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:29,595 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:53:29,595 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:53:29,611 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:29,611 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:53:29,611 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:53:29,611 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:53:29,612 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 08:53:29,613 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:53:29,613 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:53:29,613 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:53:29,613 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:53:29,613 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:53:29,614 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-08-07 08:53:29,614 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-07 08:53:29,615 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-07 08:53:29,615 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:53:29,615 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:53:29,615 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:53:29,615 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:53:29,616 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:29,616 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:53:29,616 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:29,617 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:29,617 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:29,617 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:29,617 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd, UNASSIGN 2023-08-07 08:53:29,618 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7092830e4beb39626d41e32e9a87a8fd, server=jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:29,618 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:53:29,618 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:29,618 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:29,618 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:29,618 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:53:29,618 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7092830e4beb39626d41e32e9a87a8fd, server=jenkins-hbase4.apache.org,46525,1691398396377 didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:53:29,618 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-08-07 08:53:29,619 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:53:29,619 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:53:29,620 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398439620 2023-08-07 08:53:29,620 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:53:29,621 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:29,623 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:29,624 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=7092830e4beb39626d41e32e9a87a8fd, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:29,624 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:53:29,624 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398469624 2023-08-07 08:53:29,624 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398529624 2023-08-07 08:53:29,624 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-08-07 08:53:29,627 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:53:29,629 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:53:29,629 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:29,629 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:53:29,629 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:53:29,875 WARN [Listener at localhost/33561] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-07 08:53:29,875 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:53:29,875 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:53:29,875 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:53:29,875 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 08:53:29,875 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:53:29,875 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:53:29,875 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:53:29,876 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:53:29,876 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:53:29,877 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-08-07 08:53:29,877 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-07 08:53:29,878 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-07 08:53:29,878 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:53:29,878 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:53:29,879 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:53:29,879 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:53:29,879 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:29,880 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:53:29,880 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:29,880 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:29,881 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:29,881 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:29,881 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd, UNASSIGN 2023-08-07 08:53:29,881 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7092830e4beb39626d41e32e9a87a8fd, server=jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:29,882 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:53:29,882 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:29,882 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:29,882 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:29,882 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7092830e4beb39626d41e32e9a87a8fd, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:53:29,882 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7092830e4beb39626d41e32e9a87a8fd, server=jenkins-hbase4.apache.org,46525,1691398396377 didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:53:29,882 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-08-07 08:53:29,883 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:53:29,883 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:53:29,884 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398439884 2023-08-07 08:53:29,885 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:29,885 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:53:29,888 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:29,888 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=7092830e4beb39626d41e32e9a87a8fd, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:29,888 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:53:29,888 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398469888 2023-08-07 08:53:29,888 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398529888 2023-08-07 08:53:29,888 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-08-07 08:53:29,892 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1691398409891"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1691398409891"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1691398409891"}]},"ts":"1691398409891"} 2023-08-07 08:53:29,892 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691398409891"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398409891"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398409891"}]},"ts":"1691398409891"} 2023-08-07 08:53:29,892 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691398409891"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398409891"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398409891"}]},"ts":"1691398409891"} 2023-08-07 08:53:29,898 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=e32a11e61a96e60ae4944e49d40a04ff, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a94b8879caa03f00060c928070e3e2ab, ASSIGN}] 2023-08-07 08:53:29,899 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a94b8879caa03f00060c928070e3e2ab, ASSIGN 2023-08-07 08:53:29,900 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a94b8879caa03f00060c928070e3e2ab, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,46525,1691398396377; forceNewPlan=false, retain=false 2023-08-07 08:53:29,901 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=e32a11e61a96e60ae4944e49d40a04ff, ASSIGN 2023-08-07 08:53:29,901 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=e32a11e61a96e60ae4944e49d40a04ff, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,46525,1691398396377; forceNewPlan=false, retain=false 2023-08-07 08:53:30,050 INFO [jenkins-hbase4:33357] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-08-07 08:53:30,052 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=e32a11e61a96e60ae4944e49d40a04ff, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:30,052 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691398410052"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398410052"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398410052"}]},"ts":"1691398410052"} 2023-08-07 08:53:30,054 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=31, state=RUNNABLE; OpenRegionProcedure e32a11e61a96e60ae4944e49d40a04ff, server=jenkins-hbase4.apache.org,46525,1691398396377}] 2023-08-07 08:53:30,058 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=a94b8879caa03f00060c928070e3e2ab, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:30,059 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691398410058"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398410058"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398410058"}]},"ts":"1691398410058"} 2023-08-07 08:53:30,060 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=32, state=RUNNABLE; OpenRegionProcedure a94b8879caa03f00060c928070e3e2ab, server=jenkins-hbase4.apache.org,46525,1691398396377}] 2023-08-07 08:53:30,210 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab. 2023-08-07 08:53:30,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a94b8879caa03f00060c928070e3e2ab, NAME => 'testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab.', STARTKEY => '41', ENDKEY => ''} 2023-08-07 08:53:30,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-07 08:53:30,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:30,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:53:30,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:30,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:30,212 INFO [StoreOpener-a94b8879caa03f00060c928070e3e2ab-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:30,213 DEBUG [StoreOpener-a94b8879caa03f00060c928070e3e2ab-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1 2023-08-07 08:53:30,214 DEBUG [StoreOpener-a94b8879caa03f00060c928070e3e2ab-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1 2023-08-07 08:53:30,214 INFO [StoreOpener-a94b8879caa03f00060c928070e3e2ab-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a94b8879caa03f00060c928070e3e2ab columnFamilyName cf1 2023-08-07 08:53:30,231 DEBUG [StoreOpener-a94b8879caa03f00060c928070e3e2ab-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/11698c7cdc7b42579f078d865c7b69ee.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/11698c7cdc7b42579f078d865c7b69ee-top 2023-08-07 08:53:30,236 DEBUG [StoreOpener-a94b8879caa03f00060c928070e3e2ab-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/6a5e859afc8b45509790593454f7da3d.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/6a5e859afc8b45509790593454f7da3d-top 2023-08-07 08:53:30,241 DEBUG [StoreOpener-a94b8879caa03f00060c928070e3e2ab-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/805e8b3c0672407a856b2e9957e86916.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/805e8b3c0672407a856b2e9957e86916-top 2023-08-07 08:53:30,246 DEBUG [StoreOpener-a94b8879caa03f00060c928070e3e2ab-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/bd6bd562e3df4563b70a5aad06a1c77c.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/bd6bd562e3df4563b70a5aad06a1c77c-top 2023-08-07 08:53:30,251 DEBUG [StoreOpener-a94b8879caa03f00060c928070e3e2ab-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/c41d6a566f664c6fa858730f8a1d5f30.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/c41d6a566f664c6fa858730f8a1d5f30-top 2023-08-07 08:53:30,256 DEBUG [StoreOpener-a94b8879caa03f00060c928070e3e2ab-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/ce7bda9f136841ae8c7db5220240286e.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/ce7bda9f136841ae8c7db5220240286e-top 2023-08-07 08:53:30,262 DEBUG [StoreOpener-a94b8879caa03f00060c928070e3e2ab-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/f1f56fa60c5c4350be33a4b770642c31.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/f1f56fa60c5c4350be33a4b770642c31-top 2023-08-07 08:53:30,262 INFO [StoreOpener-a94b8879caa03f00060c928070e3e2ab-1] regionserver.HStore(310): Store=a94b8879caa03f00060c928070e3e2ab/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:30,262 INFO [StoreOpener-a94b8879caa03f00060c928070e3e2ab-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:30,263 DEBUG [StoreOpener-a94b8879caa03f00060c928070e3e2ab-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2 2023-08-07 08:53:30,263 DEBUG [StoreOpener-a94b8879caa03f00060c928070e3e2ab-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2 2023-08-07 08:53:30,264 INFO [StoreOpener-a94b8879caa03f00060c928070e3e2ab-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a94b8879caa03f00060c928070e3e2ab columnFamilyName cf2 2023-08-07 08:53:30,281 DEBUG [StoreOpener-a94b8879caa03f00060c928070e3e2ab-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/04734e257fcf493c8d959eba75318ca9.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/04734e257fcf493c8d959eba75318ca9-top 2023-08-07 08:53:30,285 DEBUG [StoreOpener-a94b8879caa03f00060c928070e3e2ab-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/0a13b583104d4bf786ba47b94382f3df.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/0a13b583104d4bf786ba47b94382f3df-top 2023-08-07 08:53:30,290 DEBUG [StoreOpener-a94b8879caa03f00060c928070e3e2ab-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/a3f4c89d437d46de8b8573023aea4b4a.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/a3f4c89d437d46de8b8573023aea4b4a-top 2023-08-07 08:53:30,295 DEBUG [StoreOpener-a94b8879caa03f00060c928070e3e2ab-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/a716ca57273f49e09d9c64e312525c9b.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/a716ca57273f49e09d9c64e312525c9b-top 2023-08-07 08:53:30,299 DEBUG [StoreOpener-a94b8879caa03f00060c928070e3e2ab-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/dab875aff32b43609cfd694e172281e6.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dab875aff32b43609cfd694e172281e6-top 2023-08-07 08:53:30,304 DEBUG [StoreOpener-a94b8879caa03f00060c928070e3e2ab-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/dba9d7b0dd5743838d8dd8c952a368a9.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dba9d7b0dd5743838d8dd8c952a368a9-top 2023-08-07 08:53:30,309 DEBUG [StoreOpener-a94b8879caa03f00060c928070e3e2ab-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/dbdcd43d7530461db851264cc92dc513.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dbdcd43d7530461db851264cc92dc513-top 2023-08-07 08:53:30,309 INFO [StoreOpener-a94b8879caa03f00060c928070e3e2ab-1] regionserver.HStore(310): Store=a94b8879caa03f00060c928070e3e2ab/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:30,310 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:30,311 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:30,313 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-07 08:53:30,314 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:30,315 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a94b8879caa03f00060c928070e3e2ab; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68567573, jitterRate=0.02173645794391632}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-07 08:53:30,315 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a94b8879caa03f00060c928070e3e2ab: 2023-08-07 08:53:30,317 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab., pid=34, masterSystemTime=1691398410207 2023-08-07 08:53:30,317 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:53:30,318 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-07 08:53:30,318 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-07 08:53:30,318 INFO [RS:0;jenkins-hbase4:46525-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab. 2023-08-07 08:53:30,318 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-07 08:53:30,318 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] regionserver.HStore(1912): a94b8879caa03f00060c928070e3e2ab/cf1 is initiating minor compaction (all files) 2023-08-07 08:53:30,318 INFO [RS:0;jenkins-hbase4:46525-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab. 2023-08-07 08:53:30,318 INFO [RS:0;jenkins-hbase4:46525-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of a94b8879caa03f00060c928070e3e2ab/cf1 in testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab. 2023-08-07 08:53:30,318 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] regionserver.HStore(1912): a94b8879caa03f00060c928070e3e2ab/cf2 is initiating minor compaction (all files) 2023-08-07 08:53:30,318 INFO [RS:0;jenkins-hbase4:46525-longCompactions-0] regionserver.HRegion(2259): Starting compaction of a94b8879caa03f00060c928070e3e2ab/cf2 in testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab. 2023-08-07 08:53:30,319 INFO [RS:0;jenkins-hbase4:46525-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/ce7bda9f136841ae8c7db5220240286e.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/ce7bda9f136841ae8c7db5220240286e-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/c41d6a566f664c6fa858730f8a1d5f30.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/c41d6a566f664c6fa858730f8a1d5f30-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/11698c7cdc7b42579f078d865c7b69ee.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/11698c7cdc7b42579f078d865c7b69ee-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/f1f56fa60c5c4350be33a4b770642c31.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/f1f56fa60c5c4350be33a4b770642c31-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/bd6bd562e3df4563b70a5aad06a1c77c.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/bd6bd562e3df4563b70a5aad06a1c77c-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/6a5e859afc8b45509790593454f7da3d.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/6a5e859afc8b45509790593454f7da3d-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/805e8b3c0672407a856b2e9957e86916.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/805e8b3c0672407a856b2e9957e86916-top] into tmpdir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/.tmp, totalSize=34.9 K 2023-08-07 08:53:30,319 INFO [RS:0;jenkins-hbase4:46525-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/04734e257fcf493c8d959eba75318ca9.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/04734e257fcf493c8d959eba75318ca9-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/a716ca57273f49e09d9c64e312525c9b.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/a716ca57273f49e09d9c64e312525c9b-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/a3f4c89d437d46de8b8573023aea4b4a.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/a3f4c89d437d46de8b8573023aea4b4a-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/dab875aff32b43609cfd694e172281e6.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dab875aff32b43609cfd694e172281e6-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/dbdcd43d7530461db851264cc92dc513.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dbdcd43d7530461db851264cc92dc513-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/0a13b583104d4bf786ba47b94382f3df.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/0a13b583104d4bf786ba47b94382f3df-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/dba9d7b0dd5743838d8dd8c952a368a9.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dba9d7b0dd5743838d8dd8c952a368a9-top] into tmpdir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/.tmp, totalSize=34.9 K 2023-08-07 08:53:30,319 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] compactions.Compactor(207): Compacting ce7bda9f136841ae8c7db5220240286e.7092830e4beb39626d41e32e9a87a8fd, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1691398404950 2023-08-07 08:53:30,319 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] compactions.Compactor(207): Compacting 04734e257fcf493c8d959eba75318ca9.7092830e4beb39626d41e32e9a87a8fd, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1691398404950 2023-08-07 08:53:30,320 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] compactions.Compactor(207): Compacting c41d6a566f664c6fa858730f8a1d5f30.7092830e4beb39626d41e32e9a87a8fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1691398405152 2023-08-07 08:53:30,320 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab. 2023-08-07 08:53:30,320 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] compactions.Compactor(207): Compacting a716ca57273f49e09d9c64e312525c9b.7092830e4beb39626d41e32e9a87a8fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1691398405152 2023-08-07 08:53:30,320 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab. 2023-08-07 08:53:30,320 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff. 2023-08-07 08:53:30,321 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e32a11e61a96e60ae4944e49d40a04ff, NAME => 'testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff.', STARTKEY => '', ENDKEY => '41'} 2023-08-07 08:53:30,321 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] compactions.Compactor(207): Compacting 11698c7cdc7b42579f078d865c7b69ee.7092830e4beb39626d41e32e9a87a8fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1691398405374 2023-08-07 08:53:30,321 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=a94b8879caa03f00060c928070e3e2ab, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:30,321 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] compactions.Compactor(207): Compacting a3f4c89d437d46de8b8573023aea4b4a.7092830e4beb39626d41e32e9a87a8fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1691398405374 2023-08-07 08:53:30,321 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-07 08:53:30,321 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691398410321"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691398410321"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691398410321"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398410321"}]},"ts":"1691398410321"} 2023-08-07 08:53:30,321 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] compactions.Compactor(207): Compacting f1f56fa60c5c4350be33a4b770642c31.7092830e4beb39626d41e32e9a87a8fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1691398405585 2023-08-07 08:53:30,321 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR e32a11e61a96e60ae4944e49d40a04ff 2023-08-07 08:53:30,321 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:53:30,322 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for e32a11e61a96e60ae4944e49d40a04ff 2023-08-07 08:53:30,322 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for e32a11e61a96e60ae4944e49d40a04ff 2023-08-07 08:53:30,322 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] compactions.Compactor(207): Compacting dab875aff32b43609cfd694e172281e6.7092830e4beb39626d41e32e9a87a8fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1691398405585 2023-08-07 08:53:30,322 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] compactions.Compactor(207): Compacting bd6bd562e3df4563b70a5aad06a1c77c.7092830e4beb39626d41e32e9a87a8fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1691398405797 2023-08-07 08:53:30,325 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] compactions.Compactor(207): Compacting dbdcd43d7530461db851264cc92dc513.7092830e4beb39626d41e32e9a87a8fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1691398405797 2023-08-07 08:53:30,325 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] compactions.Compactor(207): Compacting 6a5e859afc8b45509790593454f7da3d.7092830e4beb39626d41e32e9a87a8fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1691398406010 2023-08-07 08:53:30,326 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=32 2023-08-07 08:53:30,326 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=32, state=SUCCESS; OpenRegionProcedure a94b8879caa03f00060c928070e3e2ab, server=jenkins-hbase4.apache.org,46525,1691398396377 in 263 msec 2023-08-07 08:53:30,327 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] compactions.Compactor(207): Compacting 0a13b583104d4bf786ba47b94382f3df.7092830e4beb39626d41e32e9a87a8fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1691398406010 2023-08-07 08:53:30,327 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] compactions.Compactor(207): Compacting 805e8b3c0672407a856b2e9957e86916.7092830e4beb39626d41e32e9a87a8fd, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1691398406217 2023-08-07 08:53:30,328 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a94b8879caa03f00060c928070e3e2ab, ASSIGN in 430 msec 2023-08-07 08:53:30,328 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] compactions.Compactor(207): Compacting dba9d7b0dd5743838d8dd8c952a368a9.7092830e4beb39626d41e32e9a87a8fd, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1691398406217 2023-08-07 08:53:30,331 INFO [StoreOpener-e32a11e61a96e60ae4944e49d40a04ff-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e32a11e61a96e60ae4944e49d40a04ff 2023-08-07 08:53:30,332 DEBUG [StoreOpener-e32a11e61a96e60ae4944e49d40a04ff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1 2023-08-07 08:53:30,332 DEBUG [StoreOpener-e32a11e61a96e60ae4944e49d40a04ff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1 2023-08-07 08:53:30,333 INFO [StoreOpener-e32a11e61a96e60ae4944e49d40a04ff-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e32a11e61a96e60ae4944e49d40a04ff columnFamilyName cf1 2023-08-07 08:53:30,347 INFO [RS:0;jenkins-hbase4:46525-shortCompactions-0] throttle.PressureAwareThroughputController(145): a94b8879caa03f00060c928070e3e2ab#cf1#compaction#32 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-07 08:53:30,351 INFO [RS:0;jenkins-hbase4:46525-longCompactions-0] throttle.PressureAwareThroughputController(145): a94b8879caa03f00060c928070e3e2ab#cf2#compaction#33 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-07 08:53:30,354 DEBUG [StoreOpener-e32a11e61a96e60ae4944e49d40a04ff-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/11698c7cdc7b42579f078d865c7b69ee.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/11698c7cdc7b42579f078d865c7b69ee-bottom 2023-08-07 08:53:30,358 DEBUG [StoreOpener-e32a11e61a96e60ae4944e49d40a04ff-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/6a5e859afc8b45509790593454f7da3d.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/6a5e859afc8b45509790593454f7da3d-bottom 2023-08-07 08:53:30,368 DEBUG [StoreOpener-e32a11e61a96e60ae4944e49d40a04ff-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/805e8b3c0672407a856b2e9957e86916.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/805e8b3c0672407a856b2e9957e86916-bottom 2023-08-07 08:53:30,377 DEBUG [StoreOpener-e32a11e61a96e60ae4944e49d40a04ff-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/bd6bd562e3df4563b70a5aad06a1c77c.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/bd6bd562e3df4563b70a5aad06a1c77c-bottom 2023-08-07 08:53:30,381 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/.tmp/cf2/d60e14d1e0154093973ddfccc0fc19ff as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/d60e14d1e0154093973ddfccc0fc19ff 2023-08-07 08:53:30,382 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/.tmp/cf1/0bc73330db2746338e1852f097aef741 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/0bc73330db2746338e1852f097aef741 2023-08-07 08:53:30,387 INFO [RS:0;jenkins-hbase4:46525-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a94b8879caa03f00060c928070e3e2ab/cf2 of a94b8879caa03f00060c928070e3e2ab into d60e14d1e0154093973ddfccc0fc19ff(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 08:53:30,387 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for a94b8879caa03f00060c928070e3e2ab: 2023-08-07 08:53:30,387 INFO [RS:0;jenkins-hbase4:46525-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab., storeName=a94b8879caa03f00060c928070e3e2ab/cf2, priority=9, startTime=1691398410318; duration=0sec 2023-08-07 08:53:30,387 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:53:30,393 DEBUG [StoreOpener-e32a11e61a96e60ae4944e49d40a04ff-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/c41d6a566f664c6fa858730f8a1d5f30.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/c41d6a566f664c6fa858730f8a1d5f30-bottom 2023-08-07 08:53:30,394 INFO [RS:0;jenkins-hbase4:46525-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a94b8879caa03f00060c928070e3e2ab/cf1 of a94b8879caa03f00060c928070e3e2ab into 0bc73330db2746338e1852f097aef741(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 08:53:30,394 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for a94b8879caa03f00060c928070e3e2ab: 2023-08-07 08:53:30,394 INFO [RS:0;jenkins-hbase4:46525-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab., storeName=a94b8879caa03f00060c928070e3e2ab/cf1, priority=9, startTime=1691398410317; duration=0sec 2023-08-07 08:53:30,394 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:53:30,398 DEBUG [StoreOpener-e32a11e61a96e60ae4944e49d40a04ff-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/ce7bda9f136841ae8c7db5220240286e.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/ce7bda9f136841ae8c7db5220240286e-bottom 2023-08-07 08:53:30,402 DEBUG [StoreOpener-e32a11e61a96e60ae4944e49d40a04ff-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/f1f56fa60c5c4350be33a4b770642c31.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/f1f56fa60c5c4350be33a4b770642c31-bottom 2023-08-07 08:53:30,402 INFO [StoreOpener-e32a11e61a96e60ae4944e49d40a04ff-1] regionserver.HStore(310): Store=e32a11e61a96e60ae4944e49d40a04ff/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:30,402 INFO [StoreOpener-e32a11e61a96e60ae4944e49d40a04ff-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e32a11e61a96e60ae4944e49d40a04ff 2023-08-07 08:53:30,403 DEBUG [StoreOpener-e32a11e61a96e60ae4944e49d40a04ff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2 2023-08-07 08:53:30,403 DEBUG [StoreOpener-e32a11e61a96e60ae4944e49d40a04ff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2 2023-08-07 08:53:30,403 INFO [StoreOpener-e32a11e61a96e60ae4944e49d40a04ff-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e32a11e61a96e60ae4944e49d40a04ff columnFamilyName cf2 2023-08-07 08:53:30,418 DEBUG [StoreOpener-e32a11e61a96e60ae4944e49d40a04ff-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/04734e257fcf493c8d959eba75318ca9.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/04734e257fcf493c8d959eba75318ca9-bottom 2023-08-07 08:53:30,422 DEBUG [StoreOpener-e32a11e61a96e60ae4944e49d40a04ff-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/0a13b583104d4bf786ba47b94382f3df.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/0a13b583104d4bf786ba47b94382f3df-bottom 2023-08-07 08:53:30,426 DEBUG [StoreOpener-e32a11e61a96e60ae4944e49d40a04ff-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/a3f4c89d437d46de8b8573023aea4b4a.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/a3f4c89d437d46de8b8573023aea4b4a-bottom 2023-08-07 08:53:30,431 DEBUG [StoreOpener-e32a11e61a96e60ae4944e49d40a04ff-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/a716ca57273f49e09d9c64e312525c9b.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/a716ca57273f49e09d9c64e312525c9b-bottom 2023-08-07 08:53:30,435 DEBUG [StoreOpener-e32a11e61a96e60ae4944e49d40a04ff-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/dab875aff32b43609cfd694e172281e6.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dab875aff32b43609cfd694e172281e6-bottom 2023-08-07 08:53:30,440 DEBUG [StoreOpener-e32a11e61a96e60ae4944e49d40a04ff-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/dba9d7b0dd5743838d8dd8c952a368a9.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dba9d7b0dd5743838d8dd8c952a368a9-bottom 2023-08-07 08:53:30,444 DEBUG [StoreOpener-e32a11e61a96e60ae4944e49d40a04ff-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/dbdcd43d7530461db851264cc92dc513.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dbdcd43d7530461db851264cc92dc513-bottom 2023-08-07 08:53:30,445 INFO [StoreOpener-e32a11e61a96e60ae4944e49d40a04ff-1] regionserver.HStore(310): Store=e32a11e61a96e60ae4944e49d40a04ff/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:30,445 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff 2023-08-07 08:53:30,446 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff 2023-08-07 08:53:30,448 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-07 08:53:30,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for e32a11e61a96e60ae4944e49d40a04ff 2023-08-07 08:53:30,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened e32a11e61a96e60ae4944e49d40a04ff; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67459126, jitterRate=0.005219310522079468}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-07 08:53:30,450 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for e32a11e61a96e60ae4944e49d40a04ff: 2023-08-07 08:53:30,450 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff., pid=33, masterSystemTime=1691398410207 2023-08-07 08:53:30,450 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:53:30,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-07 08:53:30,451 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-07 08:53:30,452 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-07 08:53:30,452 INFO [RS:0;jenkins-hbase4:46525-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff. 2023-08-07 08:53:30,452 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] regionserver.HStore(1912): e32a11e61a96e60ae4944e49d40a04ff/cf1 is initiating minor compaction (all files) 2023-08-07 08:53:30,452 INFO [RS:0;jenkins-hbase4:46525-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff. 2023-08-07 08:53:30,452 INFO [RS:0;jenkins-hbase4:46525-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of e32a11e61a96e60ae4944e49d40a04ff/cf1 in testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff. 2023-08-07 08:53:30,452 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] regionserver.HStore(1912): e32a11e61a96e60ae4944e49d40a04ff/cf2 is initiating minor compaction (all files) 2023-08-07 08:53:30,452 INFO [RS:0;jenkins-hbase4:46525-longCompactions-0] regionserver.HRegion(2259): Starting compaction of e32a11e61a96e60ae4944e49d40a04ff/cf2 in testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff. 2023-08-07 08:53:30,452 INFO [RS:0;jenkins-hbase4:46525-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/ce7bda9f136841ae8c7db5220240286e.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/ce7bda9f136841ae8c7db5220240286e-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/c41d6a566f664c6fa858730f8a1d5f30.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/c41d6a566f664c6fa858730f8a1d5f30-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/11698c7cdc7b42579f078d865c7b69ee.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/11698c7cdc7b42579f078d865c7b69ee-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/f1f56fa60c5c4350be33a4b770642c31.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/f1f56fa60c5c4350be33a4b770642c31-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/bd6bd562e3df4563b70a5aad06a1c77c.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/bd6bd562e3df4563b70a5aad06a1c77c-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/6a5e859afc8b45509790593454f7da3d.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/6a5e859afc8b45509790593454f7da3d-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/805e8b3c0672407a856b2e9957e86916.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/805e8b3c0672407a856b2e9957e86916-bottom] into tmpdir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/.tmp, totalSize=34.9 K 2023-08-07 08:53:30,452 INFO [RS:0;jenkins-hbase4:46525-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/04734e257fcf493c8d959eba75318ca9.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/04734e257fcf493c8d959eba75318ca9-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/a716ca57273f49e09d9c64e312525c9b.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/a716ca57273f49e09d9c64e312525c9b-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/a3f4c89d437d46de8b8573023aea4b4a.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/a3f4c89d437d46de8b8573023aea4b4a-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/dab875aff32b43609cfd694e172281e6.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dab875aff32b43609cfd694e172281e6-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/dbdcd43d7530461db851264cc92dc513.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dbdcd43d7530461db851264cc92dc513-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/0a13b583104d4bf786ba47b94382f3df.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/0a13b583104d4bf786ba47b94382f3df-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/dba9d7b0dd5743838d8dd8c952a368a9.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dba9d7b0dd5743838d8dd8c952a368a9-bottom] into tmpdir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/.tmp, totalSize=34.9 K 2023-08-07 08:53:30,453 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] compactions.Compactor(207): Compacting ce7bda9f136841ae8c7db5220240286e.7092830e4beb39626d41e32e9a87a8fd, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1691398404950 2023-08-07 08:53:30,453 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] compactions.Compactor(207): Compacting 04734e257fcf493c8d959eba75318ca9.7092830e4beb39626d41e32e9a87a8fd, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1691398404950 2023-08-07 08:53:30,453 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff. 2023-08-07 08:53:30,453 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] compactions.Compactor(207): Compacting c41d6a566f664c6fa858730f8a1d5f30.7092830e4beb39626d41e32e9a87a8fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1691398405152 2023-08-07 08:53:30,453 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff. 2023-08-07 08:53:30,453 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] compactions.Compactor(207): Compacting a716ca57273f49e09d9c64e312525c9b.7092830e4beb39626d41e32e9a87a8fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1691398405152 2023-08-07 08:53:30,454 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=e32a11e61a96e60ae4944e49d40a04ff, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:30,454 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] compactions.Compactor(207): Compacting a3f4c89d437d46de8b8573023aea4b4a.7092830e4beb39626d41e32e9a87a8fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1691398405374 2023-08-07 08:53:30,454 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691398410454"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691398410454"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691398410454"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398410454"}]},"ts":"1691398410454"} 2023-08-07 08:53:30,455 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] compactions.Compactor(207): Compacting 11698c7cdc7b42579f078d865c7b69ee.7092830e4beb39626d41e32e9a87a8fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1691398405374 2023-08-07 08:53:30,455 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] compactions.Compactor(207): Compacting dab875aff32b43609cfd694e172281e6.7092830e4beb39626d41e32e9a87a8fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1691398405585 2023-08-07 08:53:30,455 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] compactions.Compactor(207): Compacting f1f56fa60c5c4350be33a4b770642c31.7092830e4beb39626d41e32e9a87a8fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1691398405585 2023-08-07 08:53:30,455 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] compactions.Compactor(207): Compacting dbdcd43d7530461db851264cc92dc513.7092830e4beb39626d41e32e9a87a8fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1691398405797 2023-08-07 08:53:30,456 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] compactions.Compactor(207): Compacting bd6bd562e3df4563b70a5aad06a1c77c.7092830e4beb39626d41e32e9a87a8fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1691398405797 2023-08-07 08:53:30,456 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] compactions.Compactor(207): Compacting 0a13b583104d4bf786ba47b94382f3df.7092830e4beb39626d41e32e9a87a8fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1691398406010 2023-08-07 08:53:30,456 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] compactions.Compactor(207): Compacting 6a5e859afc8b45509790593454f7da3d.7092830e4beb39626d41e32e9a87a8fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1691398406010 2023-08-07 08:53:30,457 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] compactions.Compactor(207): Compacting dba9d7b0dd5743838d8dd8c952a368a9.7092830e4beb39626d41e32e9a87a8fd, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1691398406217 2023-08-07 08:53:30,457 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] compactions.Compactor(207): Compacting 805e8b3c0672407a856b2e9957e86916.7092830e4beb39626d41e32e9a87a8fd, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1691398406217 2023-08-07 08:53:30,465 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=31 2023-08-07 08:53:30,465 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=31, state=SUCCESS; OpenRegionProcedure e32a11e61a96e60ae4944e49d40a04ff, server=jenkins-hbase4.apache.org,46525,1691398396377 in 402 msec 2023-08-07 08:53:30,468 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=28 2023-08-07 08:53:30,468 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=e32a11e61a96e60ae4944e49d40a04ff, ASSIGN in 569 msec 2023-08-07 08:53:30,470 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab in 3.9870 sec 2023-08-07 08:53:30,474 INFO [RS:0;jenkins-hbase4:46525-shortCompactions-0] throttle.PressureAwareThroughputController(145): e32a11e61a96e60ae4944e49d40a04ff#cf1#compaction#34 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-07 08:53:30,475 INFO [RS:0;jenkins-hbase4:46525-longCompactions-0] throttle.PressureAwareThroughputController(145): e32a11e61a96e60ae4944e49d40a04ff#cf2#compaction#35 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-07 08:53:30,498 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/.tmp/cf1/b25d48d5ddd34593b52df78ada79f61e as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/b25d48d5ddd34593b52df78ada79f61e 2023-08-07 08:53:30,501 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/.tmp/cf2/c64c49fecd04406ea7fe7f8cb4306ae7 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/c64c49fecd04406ea7fe7f8cb4306ae7 2023-08-07 08:53:30,504 INFO [RS:0;jenkins-hbase4:46525-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e32a11e61a96e60ae4944e49d40a04ff/cf1 of e32a11e61a96e60ae4944e49d40a04ff into b25d48d5ddd34593b52df78ada79f61e(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 08:53:30,505 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for e32a11e61a96e60ae4944e49d40a04ff: 2023-08-07 08:53:30,505 INFO [RS:0;jenkins-hbase4:46525-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff., storeName=e32a11e61a96e60ae4944e49d40a04ff/cf1, priority=9, startTime=1691398410450; duration=0sec 2023-08-07 08:53:30,505 DEBUG [RS:0;jenkins-hbase4:46525-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:53:30,509 INFO [RS:0;jenkins-hbase4:46525-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e32a11e61a96e60ae4944e49d40a04ff/cf2 of e32a11e61a96e60ae4944e49d40a04ff into c64c49fecd04406ea7fe7f8cb4306ae7(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 08:53:30,509 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for e32a11e61a96e60ae4944e49d40a04ff: 2023-08-07 08:53:30,509 INFO [RS:0;jenkins-hbase4:46525-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff., storeName=e32a11e61a96e60ae4944e49d40a04ff/cf2, priority=9, startTime=1691398410451; duration=0sec 2023-08-07 08:53:30,509 DEBUG [RS:0;jenkins-hbase4:46525-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:53:30,640 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46525] ipc.CallRunner(144): callId: 241 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:54398 deadline: 1691398470639, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd. is not online on jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:30,753 DEBUG [Listener at localhost/33561] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-08-07 08:53:30,760 INFO [Listener at localhost/33561] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-08-07 08:53:30,762 DEBUG [Listener at localhost/33561] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-08-07 08:53:30,786 WARN [Listener at localhost/33561] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-07 08:53:30,788 INFO [Listener at localhost/33561] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-08-07 08:53:30,788 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitWithoutPONR 2023-08-07 08:53:30,789 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:30,792 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-08-07 08:53:30,792 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398410792"}]},"ts":"1691398410792"} 2023-08-07 08:53:30,794 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-08-07 08:53:30,796 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-08-07 08:53:30,797 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=e32a11e61a96e60ae4944e49d40a04ff, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a94b8879caa03f00060c928070e3e2ab, UNASSIGN}] 2023-08-07 08:53:30,797 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a94b8879caa03f00060c928070e3e2ab, UNASSIGN 2023-08-07 08:53:30,798 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=a94b8879caa03f00060c928070e3e2ab, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:30,798 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691398410798"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398410798"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398410798"}]},"ts":"1691398410798"} 2023-08-07 08:53:30,800 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure a94b8879caa03f00060c928070e3e2ab, server=jenkins-hbase4.apache.org,46525,1691398396377}] 2023-08-07 08:53:30,802 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=e32a11e61a96e60ae4944e49d40a04ff, UNASSIGN 2023-08-07 08:53:30,803 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=e32a11e61a96e60ae4944e49d40a04ff, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:30,803 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691398410803"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398410803"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398410803"}]},"ts":"1691398410803"} 2023-08-07 08:53:30,804 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure e32a11e61a96e60ae4944e49d40a04ff, server=jenkins-hbase4.apache.org,46525,1691398396377}] 2023-08-07 08:53:30,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-08-07 08:53:30,952 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close e32a11e61a96e60ae4944e49d40a04ff 2023-08-07 08:53:30,952 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing e32a11e61a96e60ae4944e49d40a04ff, disabling compactions & flushes 2023-08-07 08:53:30,952 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff. 2023-08-07 08:53:30,952 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff. 2023-08-07 08:53:30,952 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff. after waiting 0 ms 2023-08-07 08:53:30,952 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff. 2023-08-07 08:53:30,963 DEBUG [StoreCloser-testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/ce7bda9f136841ae8c7db5220240286e.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/ce7bda9f136841ae8c7db5220240286e-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/c41d6a566f664c6fa858730f8a1d5f30.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/c41d6a566f664c6fa858730f8a1d5f30-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/11698c7cdc7b42579f078d865c7b69ee.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/11698c7cdc7b42579f078d865c7b69ee-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/f1f56fa60c5c4350be33a4b770642c31.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/f1f56fa60c5c4350be33a4b770642c31-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/bd6bd562e3df4563b70a5aad06a1c77c.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/bd6bd562e3df4563b70a5aad06a1c77c-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/6a5e859afc8b45509790593454f7da3d.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/6a5e859afc8b45509790593454f7da3d-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/805e8b3c0672407a856b2e9957e86916.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/805e8b3c0672407a856b2e9957e86916-bottom] to archive 2023-08-07 08:53:30,965 DEBUG [StoreCloser-testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-07 08:53:30,967 DEBUG [StoreCloser-testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/ce7bda9f136841ae8c7db5220240286e.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/ce7bda9f136841ae8c7db5220240286e.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:30,968 DEBUG [StoreCloser-testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/c41d6a566f664c6fa858730f8a1d5f30.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/c41d6a566f664c6fa858730f8a1d5f30.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:30,970 DEBUG [StoreCloser-testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/11698c7cdc7b42579f078d865c7b69ee.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/11698c7cdc7b42579f078d865c7b69ee.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:30,971 DEBUG [StoreCloser-testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/f1f56fa60c5c4350be33a4b770642c31.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/f1f56fa60c5c4350be33a4b770642c31.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:30,972 DEBUG [StoreCloser-testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/bd6bd562e3df4563b70a5aad06a1c77c.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/bd6bd562e3df4563b70a5aad06a1c77c.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:30,973 DEBUG [StoreCloser-testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/6a5e859afc8b45509790593454f7da3d.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/6a5e859afc8b45509790593454f7da3d.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:30,975 DEBUG [StoreCloser-testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/805e8b3c0672407a856b2e9957e86916.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/805e8b3c0672407a856b2e9957e86916.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:30,976 DEBUG [StoreCloser-testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/04734e257fcf493c8d959eba75318ca9.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/04734e257fcf493c8d959eba75318ca9-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/a716ca57273f49e09d9c64e312525c9b.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/a716ca57273f49e09d9c64e312525c9b-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/a3f4c89d437d46de8b8573023aea4b4a.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/a3f4c89d437d46de8b8573023aea4b4a-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/dab875aff32b43609cfd694e172281e6.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dab875aff32b43609cfd694e172281e6-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/dbdcd43d7530461db851264cc92dc513.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dbdcd43d7530461db851264cc92dc513-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/0a13b583104d4bf786ba47b94382f3df.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/0a13b583104d4bf786ba47b94382f3df-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/dba9d7b0dd5743838d8dd8c952a368a9.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dba9d7b0dd5743838d8dd8c952a368a9-bottom] to archive 2023-08-07 08:53:30,977 DEBUG [StoreCloser-testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-07 08:53:30,979 DEBUG [StoreCloser-testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/04734e257fcf493c8d959eba75318ca9.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/04734e257fcf493c8d959eba75318ca9.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:30,980 DEBUG [StoreCloser-testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/a716ca57273f49e09d9c64e312525c9b.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/a716ca57273f49e09d9c64e312525c9b.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:30,981 DEBUG [StoreCloser-testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/a3f4c89d437d46de8b8573023aea4b4a.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/a3f4c89d437d46de8b8573023aea4b4a.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:30,983 DEBUG [StoreCloser-testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/dab875aff32b43609cfd694e172281e6.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/dab875aff32b43609cfd694e172281e6.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:30,984 DEBUG [StoreCloser-testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/dbdcd43d7530461db851264cc92dc513.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/dbdcd43d7530461db851264cc92dc513.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:30,985 DEBUG [StoreCloser-testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/0a13b583104d4bf786ba47b94382f3df.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/0a13b583104d4bf786ba47b94382f3df.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:30,986 DEBUG [StoreCloser-testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/dba9d7b0dd5743838d8dd8c952a368a9.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/dba9d7b0dd5743838d8dd8c952a368a9.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:30,990 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-08-07 08:53:30,991 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-07 08:53:30,991 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff. 2023-08-07 08:53:30,991 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for e32a11e61a96e60ae4944e49d40a04ff: 2023-08-07 08:53:30,992 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed e32a11e61a96e60ae4944e49d40a04ff 2023-08-07 08:53:30,992 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:30,993 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a94b8879caa03f00060c928070e3e2ab, disabling compactions & flushes 2023-08-07 08:53:30,993 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab. 2023-08-07 08:53:30,993 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab. 2023-08-07 08:53:30,993 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab. after waiting 0 ms 2023-08-07 08:53:30,993 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab. 2023-08-07 08:53:30,993 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=e32a11e61a96e60ae4944e49d40a04ff, regionState=CLOSED 2023-08-07 08:53:30,993 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691398410993"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398410993"}]},"ts":"1691398410993"} 2023-08-07 08:53:30,994 DEBUG [StoreCloser-testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/ce7bda9f136841ae8c7db5220240286e.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/ce7bda9f136841ae8c7db5220240286e-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/c41d6a566f664c6fa858730f8a1d5f30.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/c41d6a566f664c6fa858730f8a1d5f30-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/11698c7cdc7b42579f078d865c7b69ee.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/11698c7cdc7b42579f078d865c7b69ee-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/f1f56fa60c5c4350be33a4b770642c31.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/f1f56fa60c5c4350be33a4b770642c31-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/bd6bd562e3df4563b70a5aad06a1c77c.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/bd6bd562e3df4563b70a5aad06a1c77c-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/6a5e859afc8b45509790593454f7da3d.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/6a5e859afc8b45509790593454f7da3d-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/805e8b3c0672407a856b2e9957e86916.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/805e8b3c0672407a856b2e9957e86916-top] to archive 2023-08-07 08:53:30,995 DEBUG [StoreCloser-testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-07 08:53:30,997 DEBUG [StoreCloser-testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/ce7bda9f136841ae8c7db5220240286e.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/ce7bda9f136841ae8c7db5220240286e.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:30,998 DEBUG [StoreCloser-testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/c41d6a566f664c6fa858730f8a1d5f30.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/c41d6a566f664c6fa858730f8a1d5f30.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,000 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=36 2023-08-07 08:53:31,000 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure e32a11e61a96e60ae4944e49d40a04ff, server=jenkins-hbase4.apache.org,46525,1691398396377 in 193 msec 2023-08-07 08:53:31,000 DEBUG [StoreCloser-testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/11698c7cdc7b42579f078d865c7b69ee.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/11698c7cdc7b42579f078d865c7b69ee.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,002 DEBUG [StoreCloser-testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/f1f56fa60c5c4350be33a4b770642c31.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/f1f56fa60c5c4350be33a4b770642c31.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,002 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=e32a11e61a96e60ae4944e49d40a04ff, UNASSIGN in 203 msec 2023-08-07 08:53:31,003 DEBUG [StoreCloser-testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/bd6bd562e3df4563b70a5aad06a1c77c.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/bd6bd562e3df4563b70a5aad06a1c77c.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,004 DEBUG [StoreCloser-testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/6a5e859afc8b45509790593454f7da3d.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/6a5e859afc8b45509790593454f7da3d.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,005 DEBUG [StoreCloser-testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/805e8b3c0672407a856b2e9957e86916.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/805e8b3c0672407a856b2e9957e86916.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,007 DEBUG [StoreCloser-testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/04734e257fcf493c8d959eba75318ca9.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/04734e257fcf493c8d959eba75318ca9-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/a716ca57273f49e09d9c64e312525c9b.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/a716ca57273f49e09d9c64e312525c9b-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/a3f4c89d437d46de8b8573023aea4b4a.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/a3f4c89d437d46de8b8573023aea4b4a-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/dab875aff32b43609cfd694e172281e6.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dab875aff32b43609cfd694e172281e6-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/dbdcd43d7530461db851264cc92dc513.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dbdcd43d7530461db851264cc92dc513-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/0a13b583104d4bf786ba47b94382f3df.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/0a13b583104d4bf786ba47b94382f3df-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/dba9d7b0dd5743838d8dd8c952a368a9.7092830e4beb39626d41e32e9a87a8fd->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dba9d7b0dd5743838d8dd8c952a368a9-top] to archive 2023-08-07 08:53:31,008 DEBUG [StoreCloser-testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-07 08:53:31,009 DEBUG [StoreCloser-testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/04734e257fcf493c8d959eba75318ca9.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/04734e257fcf493c8d959eba75318ca9.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,010 DEBUG [StoreCloser-testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/a716ca57273f49e09d9c64e312525c9b.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/a716ca57273f49e09d9c64e312525c9b.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,012 DEBUG [StoreCloser-testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/a3f4c89d437d46de8b8573023aea4b4a.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/a3f4c89d437d46de8b8573023aea4b4a.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,013 DEBUG [StoreCloser-testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/dab875aff32b43609cfd694e172281e6.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/dab875aff32b43609cfd694e172281e6.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,014 DEBUG [StoreCloser-testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/dbdcd43d7530461db851264cc92dc513.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/dbdcd43d7530461db851264cc92dc513.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,015 DEBUG [StoreCloser-testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/0a13b583104d4bf786ba47b94382f3df.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/0a13b583104d4bf786ba47b94382f3df.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,017 DEBUG [StoreCloser-testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/dba9d7b0dd5743838d8dd8c952a368a9.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/dba9d7b0dd5743838d8dd8c952a368a9.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,020 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-08-07 08:53:31,021 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-07 08:53:31,021 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab. 2023-08-07 08:53:31,021 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a94b8879caa03f00060c928070e3e2ab: 2023-08-07 08:53:31,023 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:31,023 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=a94b8879caa03f00060c928070e3e2ab, regionState=CLOSED 2023-08-07 08:53:31,024 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691398411023"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398411023"}]},"ts":"1691398411023"} 2023-08-07 08:53:31,027 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=37 2023-08-07 08:53:31,027 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure a94b8879caa03f00060c928070e3e2ab, server=jenkins-hbase4.apache.org,46525,1691398396377 in 226 msec 2023-08-07 08:53:31,030 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=35 2023-08-07 08:53:31,030 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a94b8879caa03f00060c928070e3e2ab, UNASSIGN in 230 msec 2023-08-07 08:53:31,031 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398411030"}]},"ts":"1691398411030"} 2023-08-07 08:53:31,032 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-08-07 08:53:31,034 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-08-07 08:53:31,035 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 246 msec 2023-08-07 08:53:31,095 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-08-07 08:53:31,095 INFO [Listener at localhost/33561] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-08-07 08:53:31,096 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitWithoutPONR 2023-08-07 08:53:31,097 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure2.ProcedureExecutor(1029): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:31,099 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:31,100 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:31,100 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=40 2023-08-07 08:53:31,108 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,108 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:31,108 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff 2023-08-07 08:53:31,111 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/recovered.edits] 2023-08-07 08:53:31,111 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/a94b8879caa03f00060c928070e3e2ab, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/recovered.edits] 2023-08-07 08:53:31,111 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/e32a11e61a96e60ae4944e49d40a04ff, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/recovered.edits] 2023-08-07 08:53:31,115 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/b25d48d5ddd34593b52df78ada79f61e to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf1/b25d48d5ddd34593b52df78ada79f61e 2023-08-07 08:53:31,115 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/11698c7cdc7b42579f078d865c7b69ee to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/11698c7cdc7b42579f078d865c7b69ee 2023-08-07 08:53:31,116 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/a94b8879caa03f00060c928070e3e2ab/.regioninfo to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/a94b8879caa03f00060c928070e3e2ab/.regioninfo 2023-08-07 08:53:31,118 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/6a5e859afc8b45509790593454f7da3d to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/6a5e859afc8b45509790593454f7da3d 2023-08-07 08:53:31,119 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/c64c49fecd04406ea7fe7f8cb4306ae7 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/cf2/c64c49fecd04406ea7fe7f8cb4306ae7 2023-08-07 08:53:31,119 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/805e8b3c0672407a856b2e9957e86916 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/805e8b3c0672407a856b2e9957e86916 2023-08-07 08:53:31,120 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/a94b8879caa03f00060c928070e3e2ab/cf1/11698c7cdc7b42579f078d865c7b69ee.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/a94b8879caa03f00060c928070e3e2ab/cf1/11698c7cdc7b42579f078d865c7b69ee.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,121 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/bd6bd562e3df4563b70a5aad06a1c77c to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/bd6bd562e3df4563b70a5aad06a1c77c 2023-08-07 08:53:31,122 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/a94b8879caa03f00060c928070e3e2ab/cf1/6a5e859afc8b45509790593454f7da3d.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/a94b8879caa03f00060c928070e3e2ab/cf1/6a5e859afc8b45509790593454f7da3d.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,123 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/e32a11e61a96e60ae4944e49d40a04ff/.regioninfo to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/e32a11e61a96e60ae4944e49d40a04ff/.regioninfo 2023-08-07 08:53:31,123 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/c41d6a566f664c6fa858730f8a1d5f30 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/c41d6a566f664c6fa858730f8a1d5f30 2023-08-07 08:53:31,124 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/a94b8879caa03f00060c928070e3e2ab/cf1/805e8b3c0672407a856b2e9957e86916.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/a94b8879caa03f00060c928070e3e2ab/cf1/805e8b3c0672407a856b2e9957e86916.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,125 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/ce7bda9f136841ae8c7db5220240286e to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/ce7bda9f136841ae8c7db5220240286e 2023-08-07 08:53:31,126 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/a94b8879caa03f00060c928070e3e2ab/cf1/bd6bd562e3df4563b70a5aad06a1c77c.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/a94b8879caa03f00060c928070e3e2ab/cf1/bd6bd562e3df4563b70a5aad06a1c77c.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,127 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/e32a11e61a96e60ae4944e49d40a04ff/cf1/11698c7cdc7b42579f078d865c7b69ee.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/e32a11e61a96e60ae4944e49d40a04ff/cf1/11698c7cdc7b42579f078d865c7b69ee.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,127 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/f1f56fa60c5c4350be33a4b770642c31 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf1/f1f56fa60c5c4350be33a4b770642c31 2023-08-07 08:53:31,128 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/a94b8879caa03f00060c928070e3e2ab/cf1/c41d6a566f664c6fa858730f8a1d5f30.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/a94b8879caa03f00060c928070e3e2ab/cf1/c41d6a566f664c6fa858730f8a1d5f30.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,128 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/e32a11e61a96e60ae4944e49d40a04ff/cf1/6a5e859afc8b45509790593454f7da3d.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/e32a11e61a96e60ae4944e49d40a04ff/cf1/6a5e859afc8b45509790593454f7da3d.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,130 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/a94b8879caa03f00060c928070e3e2ab/cf1/ce7bda9f136841ae8c7db5220240286e.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/a94b8879caa03f00060c928070e3e2ab/cf1/ce7bda9f136841ae8c7db5220240286e.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,130 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/e32a11e61a96e60ae4944e49d40a04ff/cf1/805e8b3c0672407a856b2e9957e86916.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/e32a11e61a96e60ae4944e49d40a04ff/cf1/805e8b3c0672407a856b2e9957e86916.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,134 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/04734e257fcf493c8d959eba75318ca9 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/04734e257fcf493c8d959eba75318ca9 2023-08-07 08:53:31,136 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/e32a11e61a96e60ae4944e49d40a04ff/cf1/bd6bd562e3df4563b70a5aad06a1c77c.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/e32a11e61a96e60ae4944e49d40a04ff/cf1/bd6bd562e3df4563b70a5aad06a1c77c.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,136 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/a94b8879caa03f00060c928070e3e2ab/cf1/f1f56fa60c5c4350be33a4b770642c31.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/a94b8879caa03f00060c928070e3e2ab/cf1/f1f56fa60c5c4350be33a4b770642c31.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,136 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/0a13b583104d4bf786ba47b94382f3df to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/0a13b583104d4bf786ba47b94382f3df 2023-08-07 08:53:31,138 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/e32a11e61a96e60ae4944e49d40a04ff/cf1/c41d6a566f664c6fa858730f8a1d5f30.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/e32a11e61a96e60ae4944e49d40a04ff/cf1/c41d6a566f664c6fa858730f8a1d5f30.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,139 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/a3f4c89d437d46de8b8573023aea4b4a to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/a3f4c89d437d46de8b8573023aea4b4a 2023-08-07 08:53:31,141 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/e32a11e61a96e60ae4944e49d40a04ff/cf1/ce7bda9f136841ae8c7db5220240286e.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/e32a11e61a96e60ae4944e49d40a04ff/cf1/ce7bda9f136841ae8c7db5220240286e.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,141 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/a94b8879caa03f00060c928070e3e2ab/cf2/04734e257fcf493c8d959eba75318ca9.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/a94b8879caa03f00060c928070e3e2ab/cf2/04734e257fcf493c8d959eba75318ca9.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,142 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/a716ca57273f49e09d9c64e312525c9b to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/a716ca57273f49e09d9c64e312525c9b 2023-08-07 08:53:31,143 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/e32a11e61a96e60ae4944e49d40a04ff/cf1/f1f56fa60c5c4350be33a4b770642c31.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/e32a11e61a96e60ae4944e49d40a04ff/cf1/f1f56fa60c5c4350be33a4b770642c31.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,144 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/a94b8879caa03f00060c928070e3e2ab/cf2/0a13b583104d4bf786ba47b94382f3df.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/a94b8879caa03f00060c928070e3e2ab/cf2/0a13b583104d4bf786ba47b94382f3df.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,145 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dab875aff32b43609cfd694e172281e6 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dab875aff32b43609cfd694e172281e6 2023-08-07 08:53:31,147 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dba9d7b0dd5743838d8dd8c952a368a9 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dba9d7b0dd5743838d8dd8c952a368a9 2023-08-07 08:53:31,147 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/a94b8879caa03f00060c928070e3e2ab/cf2/a3f4c89d437d46de8b8573023aea4b4a.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/a94b8879caa03f00060c928070e3e2ab/cf2/a3f4c89d437d46de8b8573023aea4b4a.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,149 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/e32a11e61a96e60ae4944e49d40a04ff/cf2/04734e257fcf493c8d959eba75318ca9.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/e32a11e61a96e60ae4944e49d40a04ff/cf2/04734e257fcf493c8d959eba75318ca9.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,149 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/a94b8879caa03f00060c928070e3e2ab/cf2/a716ca57273f49e09d9c64e312525c9b.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/a94b8879caa03f00060c928070e3e2ab/cf2/a716ca57273f49e09d9c64e312525c9b.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,150 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dbdcd43d7530461db851264cc92dc513 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/cf2/dbdcd43d7530461db851264cc92dc513 2023-08-07 08:53:31,152 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/e32a11e61a96e60ae4944e49d40a04ff/cf2/0a13b583104d4bf786ba47b94382f3df.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/e32a11e61a96e60ae4944e49d40a04ff/cf2/0a13b583104d4bf786ba47b94382f3df.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,152 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/a94b8879caa03f00060c928070e3e2ab/cf2/dab875aff32b43609cfd694e172281e6.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/a94b8879caa03f00060c928070e3e2ab/cf2/dab875aff32b43609cfd694e172281e6.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,154 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/e32a11e61a96e60ae4944e49d40a04ff/cf2/a3f4c89d437d46de8b8573023aea4b4a.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/e32a11e61a96e60ae4944e49d40a04ff/cf2/a3f4c89d437d46de8b8573023aea4b4a.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,154 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/a94b8879caa03f00060c928070e3e2ab/cf2/dba9d7b0dd5743838d8dd8c952a368a9.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/a94b8879caa03f00060c928070e3e2ab/cf2/dba9d7b0dd5743838d8dd8c952a368a9.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,154 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/recovered.edits/85.seqid to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd/recovered.edits/85.seqid 2023-08-07 08:53:31,155 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,156 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/e32a11e61a96e60ae4944e49d40a04ff/cf2/a716ca57273f49e09d9c64e312525c9b.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/e32a11e61a96e60ae4944e49d40a04ff/cf2/a716ca57273f49e09d9c64e312525c9b.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,156 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/a94b8879caa03f00060c928070e3e2ab/cf2/dbdcd43d7530461db851264cc92dc513.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/a94b8879caa03f00060c928070e3e2ab/cf2/dbdcd43d7530461db851264cc92dc513.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,157 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/e32a11e61a96e60ae4944e49d40a04ff/cf2/dab875aff32b43609cfd694e172281e6.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/e32a11e61a96e60ae4944e49d40a04ff/cf2/dab875aff32b43609cfd694e172281e6.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,158 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/e32a11e61a96e60ae4944e49d40a04ff/cf2/dba9d7b0dd5743838d8dd8c952a368a9.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/e32a11e61a96e60ae4944e49d40a04ff/cf2/dba9d7b0dd5743838d8dd8c952a368a9.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,158 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/0bc73330db2746338e1852f097aef741 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf1/0bc73330db2746338e1852f097aef741 2023-08-07 08:53:31,160 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/e32a11e61a96e60ae4944e49d40a04ff/cf2/dbdcd43d7530461db851264cc92dc513.7092830e4beb39626d41e32e9a87a8fd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/e32a11e61a96e60ae4944e49d40a04ff/cf2/dbdcd43d7530461db851264cc92dc513.7092830e4beb39626d41e32e9a87a8fd 2023-08-07 08:53:31,161 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/d60e14d1e0154093973ddfccc0fc19ff to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/cf2/d60e14d1e0154093973ddfccc0fc19ff 2023-08-07 08:53:31,163 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/recovered.edits/90.seqid to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff/recovered.edits/90.seqid 2023-08-07 08:53:31,164 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/e32a11e61a96e60ae4944e49d40a04ff 2023-08-07 08:53:31,165 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/recovered.edits/90.seqid to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab/recovered.edits/90.seqid 2023-08-07 08:53:31,165 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitWithoutPONR/a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:31,165 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-08-07 08:53:31,168 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:31,171 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-08-07 08:53:31,173 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-08-07 08:53:31,175 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:31,175 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-08-07 08:53:31,175 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691398411175"}]},"ts":"9223372036854775807"} 2023-08-07 08:53:31,175 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691398411175"}]},"ts":"9223372036854775807"} 2023-08-07 08:53:31,175 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691398411175"}]},"ts":"9223372036854775807"} 2023-08-07 08:53:31,177 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-08-07 08:53:31,177 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 7092830e4beb39626d41e32e9a87a8fd, NAME => 'testSplitWithoutPONR,,1691398404445.7092830e4beb39626d41e32e9a87a8fd.', STARTKEY => '', ENDKEY => ''}, {ENCODED => e32a11e61a96e60ae4944e49d40a04ff, NAME => 'testSplitWithoutPONR,,1691398406480.e32a11e61a96e60ae4944e49d40a04ff.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => a94b8879caa03f00060c928070e3e2ab, NAME => 'testSplitWithoutPONR,41,1691398406480.a94b8879caa03f00060c928070e3e2ab.', STARTKEY => '41', ENDKEY => ''}] 2023-08-07 08:53:31,177 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-08-07 08:53:31,178 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1691398411177"}]},"ts":"9223372036854775807"} 2023-08-07 08:53:31,179 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-08-07 08:53:31,181 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:31,183 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 85 msec 2023-08-07 08:53:31,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=40 2023-08-07 08:53:31,202 INFO [Listener at localhost/33561] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-08-07 08:53:31,218 INFO [Listener at localhost/33561] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=397 (was 391) Potentially hanging thread: hconnection-0x36ae5be-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2e450bc6-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_63952265_17 at /127.0.0.1:53284 [Waiting for operation #28] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/cluster_383c75a2-fb83-c721-ca60-8c2ce8eb403c/dfs/data/data3/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_63952265_17 at /127.0.0.1:33698 [Waiting for operation #29] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_63952265_17 at /127.0.0.1:53288 [Waiting for operation #28] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/cluster_383c75a2-fb83-c721-ca60-8c2ce8eb403c/dfs/data/data4/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_63952265_17 at /127.0.0.1:50888 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x36ae5be-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x36ae5be-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2e450bc6-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x36ae5be-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/cluster_383c75a2-fb83-c721-ca60-8c2ce8eb403c/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_63952265_17 at /127.0.0.1:57054 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_63952265_17 at /127.0.0.1:56920 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_63952265_17 at /127.0.0.1:49878 [Waiting for operation #24] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/cluster_383c75a2-fb83-c721-ca60-8c2ce8eb403c/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x36ae5be-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x36ae5be-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_63952265_17 at /127.0.0.1:53904 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_63952265_17 at /127.0.0.1:50146 [Waiting for operation #26] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_63952265_17 at /127.0.0.1:50860 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_63952265_17 at /127.0.0.1:56870 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_63952265_17 at /127.0.0.1:50864 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_63952265_17 at /127.0.0.1:56910 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_63952265_17 at /127.0.0.1:49910 [Waiting for operation #25] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_63952265_17 at /127.0.0.1:50878 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_63952265_17 at /127.0.0.1:53994 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=873 (was 835) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=408 (was 374) - SystemLoadAverage LEAK? -, ProcessCount=174 (was 174), AvailableMemoryMB=2016 (was 2014) - AvailableMemoryMB LEAK? - 2023-08-07 08:53:31,236 INFO [Listener at localhost/33561] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=397, OpenFileDescriptor=873, MaxFileDescriptor=60000, SystemLoadAverage=408, ProcessCount=174, AvailableMemoryMB=2015 2023-08-07 08:53:31,236 WARN [Listener at localhost/33561] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-07 08:53:31,239 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-08-07 08:53:31,240 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-08-07 08:53:31,241 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(1029): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:53:31,244 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-08-07 08:53:31,244 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-07 08:53:31,246 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:31,247 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e empty. 2023-08-07 08:53:31,247 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:31,247 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-08-07 08:53:31,259 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-08-07 08:53:31,261 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7675): creating {ENCODED => fd83112e7ec055aba3e5a554dab2841e, NAME => 'testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp 2023-08-07 08:53:31,269 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:53:31,270 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1604): Closing fd83112e7ec055aba3e5a554dab2841e, disabling compactions & flushes 2023-08-07 08:53:31,270 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. 2023-08-07 08:53:31,270 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. 2023-08-07 08:53:31,270 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. after waiting 0 ms 2023-08-07 08:53:31,270 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. 2023-08-07 08:53:31,270 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. 2023-08-07 08:53:31,270 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1558): Region close journal for fd83112e7ec055aba3e5a554dab2841e: 2023-08-07 08:53:31,272 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-08-07 08:53:31,273 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691398411273"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398411273"}]},"ts":"1691398411273"} 2023-08-07 08:53:31,275 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-07 08:53:31,276 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-07 08:53:31,276 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398411276"}]},"ts":"1691398411276"} 2023-08-07 08:53:31,277 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-08-07 08:53:31,280 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-07 08:53:31,281 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-07 08:53:31,281 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-07 08:53:31,281 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-07 08:53:31,281 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-07 08:53:31,281 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN}] 2023-08-07 08:53:31,283 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN 2023-08-07 08:53:31,284 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34559,1691398396467; forceNewPlan=false, retain=false 2023-08-07 08:53:31,434 INFO [jenkins-hbase4:33357] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-07 08:53:31,435 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=fd83112e7ec055aba3e5a554dab2841e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:31,435 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691398411435"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398411435"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398411435"}]},"ts":"1691398411435"} 2023-08-07 08:53:31,437 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure fd83112e7ec055aba3e5a554dab2841e, server=jenkins-hbase4.apache.org,34559,1691398396467}] 2023-08-07 08:53:31,597 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. 2023-08-07 08:53:31,597 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => fd83112e7ec055aba3e5a554dab2841e, NAME => 'testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.', STARTKEY => '', ENDKEY => ''} 2023-08-07 08:53:31,597 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-07 08:53:31,598 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:31,598 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:53:31,598 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:31,598 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:31,599 INFO [StoreOpener-fd83112e7ec055aba3e5a554dab2841e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:31,601 DEBUG [StoreOpener-fd83112e7ec055aba3e5a554dab2841e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1 2023-08-07 08:53:31,601 DEBUG [StoreOpener-fd83112e7ec055aba3e5a554dab2841e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1 2023-08-07 08:53:31,601 INFO [StoreOpener-fd83112e7ec055aba3e5a554dab2841e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fd83112e7ec055aba3e5a554dab2841e columnFamilyName cf1 2023-08-07 08:53:31,602 INFO [StoreOpener-fd83112e7ec055aba3e5a554dab2841e-1] regionserver.HStore(310): Store=fd83112e7ec055aba3e5a554dab2841e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:31,602 INFO [StoreOpener-fd83112e7ec055aba3e5a554dab2841e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:31,603 DEBUG [StoreOpener-fd83112e7ec055aba3e5a554dab2841e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2 2023-08-07 08:53:31,603 DEBUG [StoreOpener-fd83112e7ec055aba3e5a554dab2841e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2 2023-08-07 08:53:31,603 INFO [StoreOpener-fd83112e7ec055aba3e5a554dab2841e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fd83112e7ec055aba3e5a554dab2841e columnFamilyName cf2 2023-08-07 08:53:31,604 INFO [StoreOpener-fd83112e7ec055aba3e5a554dab2841e-1] regionserver.HStore(310): Store=fd83112e7ec055aba3e5a554dab2841e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:31,605 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:31,605 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:31,607 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-07 08:53:31,608 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:31,610 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 08:53:31,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened fd83112e7ec055aba3e5a554dab2841e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65739247, jitterRate=-0.020408883690834045}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-07 08:53:31,610 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for fd83112e7ec055aba3e5a554dab2841e: 2023-08-07 08:53:31,611 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e., pid=43, masterSystemTime=1691398411589 2023-08-07 08:53:31,613 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. 2023-08-07 08:53:31,613 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. 2023-08-07 08:53:31,614 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=fd83112e7ec055aba3e5a554dab2841e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:31,614 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691398411613"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691398411613"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691398411613"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398411613"}]},"ts":"1691398411613"} 2023-08-07 08:53:31,618 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=42 2023-08-07 08:53:31,618 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure fd83112e7ec055aba3e5a554dab2841e, server=jenkins-hbase4.apache.org,34559,1691398396467 in 178 msec 2023-08-07 08:53:31,620 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=41 2023-08-07 08:53:31,620 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN in 337 msec 2023-08-07 08:53:31,620 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-07 08:53:31,621 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398411620"}]},"ts":"1691398411620"} 2023-08-07 08:53:31,622 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-08-07 08:53:31,632 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-08-07 08:53:31,634 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 393 msec 2023-08-07 08:53:31,752 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:53:31,754 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-08-07 08:53:31,754 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-08-07 08:53:31,754 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:31,755 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-08-07 08:53:31,755 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-08-07 08:53:31,755 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:31,755 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:31,756 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:31,756 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:31,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:31,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:31,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:31,756 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:31,756 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:31,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:31,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:31,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:31,757 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:31,757 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:53:31,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:31,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:31,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:31,758 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:31,758 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:31,758 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:31,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:31,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:31,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:31,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:31,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:31,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:31,758 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-07 08:53:31,759 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-07 08:53:31,759 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-07 08:53:31,759 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-07 08:53:31,759 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-07 08:53:31,759 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-07 08:53:31,759 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-07 08:53:31,759 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-07 08:53:31,759 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-07 08:53:31,759 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:31,759 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:31,759 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-07 08:53:31,759 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-07 08:53:31,759 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:31,759 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-07 08:53:31,760 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-07 08:53:31,759 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-07 08:53:31,760 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-07 08:53:31,760 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-07 08:53:31,760 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. 2023-08-07 08:53:31,760 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:53:31,760 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. started... 2023-08-07 08:53:31,760 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:53:31,761 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fd83112e7ec055aba3e5a554dab2841e 2/2 column families, dataSize=120 B heapSize=928 B 2023-08-07 08:53:31,762 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:31,763 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:31,763 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:31,763 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:31,763 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:31,763 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:31,777 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/.tmp/cf1/a2e4aab8fc0f4f01bed8c08d5e21e5ad 2023-08-07 08:53:31,792 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/.tmp/cf2/f5df38fe79b345d783c503b0e7ebbd90 2023-08-07 08:53:31,797 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/.tmp/cf1/a2e4aab8fc0f4f01bed8c08d5e21e5ad as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/a2e4aab8fc0f4f01bed8c08d5e21e5ad 2023-08-07 08:53:31,802 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/a2e4aab8fc0f4f01bed8c08d5e21e5ad, entries=2, sequenceid=6, filesize=4.8 K 2023-08-07 08:53:31,803 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/.tmp/cf2/f5df38fe79b345d783c503b0e7ebbd90 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/f5df38fe79b345d783c503b0e7ebbd90 2023-08-07 08:53:31,809 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/f5df38fe79b345d783c503b0e7ebbd90, entries=2, sequenceid=6, filesize=4.8 K 2023-08-07 08:53:31,810 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for fd83112e7ec055aba3e5a554dab2841e in 49ms, sequenceid=6, compaction requested=false 2023-08-07 08:53:31,810 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-08-07 08:53:31,811 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fd83112e7ec055aba3e5a554dab2841e: 2023-08-07 08:53:31,811 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. 2023-08-07 08:53:31,811 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:53:31,811 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:53:31,811 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:31,811 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-07 08:53:31,811 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-07 08:53:31,814 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:31,814 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:31,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:31,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:31,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:31,814 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:31,814 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:31,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:31,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:31,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-07 08:53:31,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:31,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:31,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:31,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:31,817 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-08-07 08:53:31,817 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@41318df[Count = 0] remaining members to acquire global barrier 2023-08-07 08:53:31,817 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-08-07 08:53:31,817 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:31,819 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:31,819 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:31,819 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:31,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:31,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:31,819 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:31,819 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:53:31,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:31,819 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-07 08:53:31,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:31,819 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-07 08:53:31,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:31,819 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:53:31,819 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-07 08:53:31,820 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-07 08:53:31,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:31,820 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:53:31,820 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-07 08:53:31,820 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-07 08:53:31,820 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:53:31,821 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-07 08:53:31,821 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:31,821 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-07 08:53:31,822 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:31,822 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-07 08:53:31,822 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-07 08:53:31,822 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:31,822 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:31,822 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-07 08:53:31,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:31,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:31,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:31,822 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-07 08:53:31,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:31,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:31,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-07 08:53:31,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:31,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:31,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:31,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:31,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-07 08:53:31,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:31,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:31,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:31,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,34559,1691398396467': 2023-08-07 08:53:31,827 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-08-07 08:53:31,827 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-08-07 08:53:31,827 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:53:31,827 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:53:31,827 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-08-07 08:53:31,827 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:53:31,831 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:31,831 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:31,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:31,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:31,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:31,831 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:31,831 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:31,831 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:31,831 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:31,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:31,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:31,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:31,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:31,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:31,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:31,831 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:31,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:31,832 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:31,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:31,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:31,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:31,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:31,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:31,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:31,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:31,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:31,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-07 08:53:31,833 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:31,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:31,833 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:31,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:31,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-07 08:53:31,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:31,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:31,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:31,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:31,835 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:31,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-07 08:53:31,836 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:31,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:31,836 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:31,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:31,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:31,839 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:31,839 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:31,839 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:31,839 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:31,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:31,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:31,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:31,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:31,839 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:31,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:31,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:31,839 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:31,839 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:31,840 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:31,840 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:31,840 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:31,840 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:31,840 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-08-07 08:53:31,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:31,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:31,839 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:31,840 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:53:31,840 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:31,840 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:31,840 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:31,840 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:31,840 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:31,840 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-08-07 08:53:31,840 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:31,841 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:53:31,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:31,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:31,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:31,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:31,941 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:53:31,942 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:53:31,958 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:53:31,960 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-08-07 08:53:31,960 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-08-07 08:53:31,960 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-08-07 08:53:31,960 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:31,960 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-08-07 08:53:31,960 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-08-07 08:53:31,960 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:31,960 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:31,962 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:31,962 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:31,962 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:31,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:31,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:31,962 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:31,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:31,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:31,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:31,962 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:31,962 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:53:31,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:31,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:31,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:31,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:31,963 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:31,963 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:31,963 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:31,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:31,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:31,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:31,964 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-07 08:53:31,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:31,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:31,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:31,964 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-07 08:53:31,964 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-07 08:53:31,964 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-07 08:53:31,964 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-07 08:53:31,964 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-07 08:53:31,964 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:31,964 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-07 08:53:31,966 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-07 08:53:31,966 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-07 08:53:31,966 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-07 08:53:31,966 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:31,966 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-07 08:53:31,967 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. 2023-08-07 08:53:31,966 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-07 08:53:31,966 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:53:31,967 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. started... 2023-08-07 08:53:31,967 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:53:31,967 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-07 08:53:31,967 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-07 08:53:31,967 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-07 08:53:31,967 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-07 08:53:31,967 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fd83112e7ec055aba3e5a554dab2841e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:53:31,967 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:31,967 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-07 08:53:31,967 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-07 08:53:31,967 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-07 08:53:31,968 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:31,970 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:31,970 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:31,970 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:31,971 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:31,971 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:31,978 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/.tmp/cf1/2489c9607b2645d5845bd72a39dbb77b 2023-08-07 08:53:31,993 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/.tmp/cf2/acf834d7b8cb41aea32f6e17233515cf 2023-08-07 08:53:31,998 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/.tmp/cf1/2489c9607b2645d5845bd72a39dbb77b as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/2489c9607b2645d5845bd72a39dbb77b 2023-08-07 08:53:32,003 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/2489c9607b2645d5845bd72a39dbb77b, entries=10, sequenceid=19, filesize=5.0 K 2023-08-07 08:53:32,004 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/.tmp/cf2/acf834d7b8cb41aea32f6e17233515cf as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/acf834d7b8cb41aea32f6e17233515cf 2023-08-07 08:53:32,010 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/acf834d7b8cb41aea32f6e17233515cf, entries=10, sequenceid=19, filesize=5.0 K 2023-08-07 08:53:32,011 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for fd83112e7ec055aba3e5a554dab2841e in 44ms, sequenceid=19, compaction requested=false 2023-08-07 08:53:32,011 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fd83112e7ec055aba3e5a554dab2841e: 2023-08-07 08:53:32,011 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. 2023-08-07 08:53:32,011 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:53:32,011 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:53:32,011 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,011 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-07 08:53:32,011 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-07 08:53:32,013 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,013 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:32,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:32,014 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,014 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:32,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:32,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:32,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-07 08:53:32,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:32,016 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-08-07 08:53:32,016 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@162c1511[Count = 0] remaining members to acquire global barrier 2023-08-07 08:53:32,016 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-08-07 08:53:32,017 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,018 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,018 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,018 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,018 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,018 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:53:32,018 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-07 08:53:32,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,019 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-07 08:53:32,018 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-07 08:53:32,019 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:53:32,019 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-07 08:53:32,019 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-07 08:53:32,019 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:53:32,019 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-07 08:53:32,019 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:53:32,021 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-07 08:53:32,021 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:32,021 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-07 08:53:32,025 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,025 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-07 08:53:32,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:32,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:32,025 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:32,026 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-07 08:53:32,025 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-07 08:53:32,026 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:32,026 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-07 08:53:32,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:32,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:32,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-07 08:53:32,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:32,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-07 08:53:32,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,34559,1691398396467': 2023-08-07 08:53:32,031 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-08-07 08:53:32,031 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-08-07 08:53:32,031 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:53:32,031 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:53:32,031 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-08-07 08:53:32,031 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:53:32,033 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,033 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,033 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,033 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:32,033 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:32,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:32,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,033 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:32,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,033 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:32,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,033 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:32,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:32,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:32,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:32,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:32,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:32,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:32,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,034 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-07 08:53:32,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,035 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:32,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-07 08:53:32,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,037 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,038 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:32,038 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-07 08:53:32,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,042 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,042 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:32,042 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,042 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:32,042 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:32,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:32,042 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:32,042 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-08-07 08:53:32,042 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:32,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:32,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:32,042 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,042 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:32,042 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:32,042 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:32,042 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:53:32,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:32,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:32,042 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:32,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:32,042 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,043 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,043 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-08-07 08:53:32,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:32,043 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:53:32,043 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,043 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,043 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,043 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:32,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:32,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:32,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:32,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:32,143 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:53:32,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:53:32,161 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:53:32,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-08-07 08:53:32,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-08-07 08:53:32,162 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-08-07 08:53:32,163 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:32,163 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-08-07 08:53:32,163 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-08-07 08:53:32,163 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,163 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:32,164 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:32,164 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,164 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:32,164 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:32,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:32,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:32,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:32,165 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,165 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:53:32,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:32,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:32,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:32,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:32,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:32,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:32,166 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,166 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,166 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:32,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:32,166 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-07 08:53:32,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:32,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:32,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:32,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:32,166 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-07 08:53:32,166 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-07 08:53:32,166 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-07 08:53:32,167 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-07 08:53:32,167 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-07 08:53:32,167 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-07 08:53:32,167 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-07 08:53:32,167 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-07 08:53:32,167 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:32,167 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-07 08:53:32,167 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-07 08:53:32,167 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-07 08:53:32,167 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:32,167 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:32,167 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-07 08:53:32,167 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-07 08:53:32,167 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-07 08:53:32,167 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-07 08:53:32,167 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-07 08:53:32,167 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-07 08:53:32,167 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-07 08:53:32,167 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:53:32,167 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:53:32,167 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. 2023-08-07 08:53:32,168 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. started... 2023-08-07 08:53:32,168 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fd83112e7ec055aba3e5a554dab2841e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:53:32,170 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,171 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,171 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,171 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:32,171 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,171 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:32,178 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/.tmp/cf1/6b8a137fb9644c108395222874065f46 2023-08-07 08:53:32,205 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/.tmp/cf2/da3a5ec77c6c42c4a4be4511e65db6bc 2023-08-07 08:53:32,211 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/.tmp/cf1/6b8a137fb9644c108395222874065f46 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/6b8a137fb9644c108395222874065f46 2023-08-07 08:53:32,217 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/6b8a137fb9644c108395222874065f46, entries=10, sequenceid=32, filesize=5.0 K 2023-08-07 08:53:32,218 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/.tmp/cf2/da3a5ec77c6c42c4a4be4511e65db6bc as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/da3a5ec77c6c42c4a4be4511e65db6bc 2023-08-07 08:53:32,223 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/da3a5ec77c6c42c4a4be4511e65db6bc, entries=10, sequenceid=32, filesize=5.0 K 2023-08-07 08:53:32,224 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for fd83112e7ec055aba3e5a554dab2841e in 56ms, sequenceid=32, compaction requested=true 2023-08-07 08:53:32,224 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fd83112e7ec055aba3e5a554dab2841e: 2023-08-07 08:53:32,224 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. 2023-08-07 08:53:32,224 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:53:32,224 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:53:32,224 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,224 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-07 08:53:32,224 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-07 08:53:32,226 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,226 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:32,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:32,227 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,227 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:32,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:32,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:32,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-07 08:53:32,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:32,229 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-08-07 08:53:32,229 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@43480071[Count = 0] remaining members to acquire global barrier 2023-08-07 08:53:32,229 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-08-07 08:53:32,229 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,230 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,230 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,230 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,230 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-07 08:53:32,231 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-07 08:53:32,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,231 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:53:32,231 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-07 08:53:32,231 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-07 08:53:32,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,230 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,231 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-07 08:53:32,231 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:53:32,231 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-07 08:53:32,231 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:53:32,231 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:53:32,234 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-07 08:53:32,234 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:32,234 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-07 08:53:32,235 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-07 08:53:32,235 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,235 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-07 08:53:32,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,235 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:32,235 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:32,235 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-07 08:53:32,235 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-07 08:53:32,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:32,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:32,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:32,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:32,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-07 08:53:32,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:32,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-07 08:53:32,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,34559,1691398396467': 2023-08-07 08:53:32,240 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-08-07 08:53:32,240 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-08-07 08:53:32,240 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:53:32,240 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:53:32,240 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-08-07 08:53:32,240 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:53:32,242 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,242 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,242 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,242 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:32,242 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,242 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:32,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:32,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,243 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:32,242 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:32,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:32,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:32,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:32,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:32,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:32,243 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:32,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:32,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,244 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-07 08:53:32,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:32,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-07 08:53:32,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,246 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:32,247 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-07 08:53:32,247 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,254 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,254 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:32,254 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:32,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:32,254 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:32,254 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:32,254 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,254 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:32,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:32,254 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:32,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:32,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:32,254 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-08-07 08:53:32,255 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:53:32,254 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:32,254 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,254 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:32,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:32,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:32,255 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:32,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:32,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:32,255 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-08-07 08:53:32,255 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,255 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:53:32,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:32,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:32,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:32,255 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:32,256 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,256 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,256 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,256 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,355 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:53:32,357 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:53:32,372 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:53:32,374 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-08-07 08:53:32,374 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-08-07 08:53:32,374 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-08-07 08:53:32,374 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:32,375 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-08-07 08:53:32,375 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-08-07 08:53:32,375 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,375 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:32,376 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:32,377 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,376 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:32,377 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:32,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:32,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:32,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:32,377 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,377 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:53:32,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:32,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:32,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:32,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:32,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:32,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:32,378 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,378 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,378 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:32,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:32,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:32,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:32,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:32,378 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-07 08:53:32,378 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-07 08:53:32,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:32,379 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-07 08:53:32,379 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-07 08:53:32,379 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-07 08:53:32,379 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-07 08:53:32,379 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-07 08:53:32,379 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-07 08:53:32,379 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-07 08:53:32,379 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-07 08:53:32,379 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:32,379 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:32,379 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-07 08:53:32,379 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-07 08:53:32,379 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-07 08:53:32,379 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-07 08:53:32,379 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-07 08:53:32,379 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-07 08:53:32,379 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-07 08:53:32,379 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-07 08:53:32,380 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:32,380 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-07 08:53:32,380 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:53:32,380 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:53:32,380 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. 2023-08-07 08:53:32,380 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. started... 2023-08-07 08:53:32,380 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fd83112e7ec055aba3e5a554dab2841e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:53:32,381 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,381 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,381 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,382 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:32,382 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,382 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:32,391 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/.tmp/cf1/ac88c845249d46e49f0532262366d399 2023-08-07 08:53:32,405 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/.tmp/cf2/1ca97b2be41f490591a5050a91d41fa9 2023-08-07 08:53:32,410 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/.tmp/cf1/ac88c845249d46e49f0532262366d399 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/ac88c845249d46e49f0532262366d399 2023-08-07 08:53:32,414 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/ac88c845249d46e49f0532262366d399, entries=10, sequenceid=45, filesize=5.0 K 2023-08-07 08:53:32,415 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/.tmp/cf2/1ca97b2be41f490591a5050a91d41fa9 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/1ca97b2be41f490591a5050a91d41fa9 2023-08-07 08:53:32,419 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/1ca97b2be41f490591a5050a91d41fa9, entries=10, sequenceid=45, filesize=5.0 K 2023-08-07 08:53:32,420 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for fd83112e7ec055aba3e5a554dab2841e in 40ms, sequenceid=45, compaction requested=true 2023-08-07 08:53:32,421 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fd83112e7ec055aba3e5a554dab2841e: 2023-08-07 08:53:32,421 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. 2023-08-07 08:53:32,421 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:53:32,421 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:53:32,421 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,421 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-07 08:53:32,421 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-07 08:53:32,422 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,422 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:32,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:32,423 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,423 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:32,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:32,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:32,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-07 08:53:32,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:32,425 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-08-07 08:53:32,425 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6e2923f9[Count = 0] remaining members to acquire global barrier 2023-08-07 08:53:32,425 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-08-07 08:53:32,425 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,427 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,427 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,427 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,427 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,427 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-07 08:53:32,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,428 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-07 08:53:32,427 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:53:32,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,428 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:53:32,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,428 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-07 08:53:32,428 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-07 08:53:32,428 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-07 08:53:32,428 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-07 08:53:32,428 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:53:32,428 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:53:32,429 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,429 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-07 08:53:32,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:32,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:32,429 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:32,429 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-07 08:53:32,430 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-07 08:53:32,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:32,430 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:32,430 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-07 08:53:32,430 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-07 08:53:32,430 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:32,430 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-07 08:53:32,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:32,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-07 08:53:32,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:32,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-07 08:53:32,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,34559,1691398396467': 2023-08-07 08:53:32,433 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-08-07 08:53:32,433 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-08-07 08:53:32,433 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:53:32,433 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:53:32,433 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-08-07 08:53:32,433 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:53:32,435 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,435 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,435 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,435 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,435 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:32,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,435 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:32,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,435 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:32,435 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:32,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:32,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:32,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:32,436 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:32,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:32,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:32,436 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:32,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:32,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-07 08:53:32,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:32,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-07 08:53:32,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,438 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,438 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:32,439 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-07 08:53:32,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,442 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,442 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:32,442 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:32,442 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:32,442 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:32,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:32,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:32,442 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-08-07 08:53:32,442 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:53:32,442 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:32,442 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:32,442 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:32,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:32,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:32,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:32,442 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:32,443 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-08-07 08:53:32,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:32,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:32,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:32,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:32,442 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:32,443 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,443 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:53:32,443 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:32,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:32,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:32,443 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,443 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,443 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,443 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,443 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,443 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,543 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:53:32,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:53:32,560 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:53:32,562 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-08-07 08:53:32,562 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-08-07 08:53:32,562 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-08-07 08:53:32,562 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:32,562 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-08-07 08:53:32,562 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-08-07 08:53:32,562 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,562 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:32,564 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:32,564 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:32,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:32,564 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:32,564 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:32,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:32,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:32,564 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:32,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:32,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:32,564 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:53:32,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:32,565 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:32,565 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,565 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:32,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:32,565 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-07 08:53:32,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:32,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:32,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:32,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:32,566 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-07 08:53:32,565 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-07 08:53:32,566 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-07 08:53:32,566 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-07 08:53:32,566 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-07 08:53:32,566 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-07 08:53:32,566 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-07 08:53:32,566 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:32,566 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-07 08:53:32,566 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-07 08:53:32,566 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-07 08:53:32,566 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-07 08:53:32,566 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:32,566 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:32,566 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-07 08:53:32,566 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-07 08:53:32,566 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-07 08:53:32,566 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-07 08:53:32,566 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-07 08:53:32,566 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-07 08:53:32,566 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-07 08:53:32,567 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:53:32,567 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:53:32,567 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. 2023-08-07 08:53:32,567 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. started... 2023-08-07 08:53:32,567 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fd83112e7ec055aba3e5a554dab2841e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:53:32,568 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,569 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,569 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,569 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:32,569 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,569 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:32,577 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/.tmp/cf1/77fb8078d2014f87887d92cddcb5d75e 2023-08-07 08:53:32,590 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/.tmp/cf2/de611d4e2cf544928b1f9d901618602a 2023-08-07 08:53:32,595 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/.tmp/cf1/77fb8078d2014f87887d92cddcb5d75e as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/77fb8078d2014f87887d92cddcb5d75e 2023-08-07 08:53:32,600 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/77fb8078d2014f87887d92cddcb5d75e, entries=10, sequenceid=58, filesize=5.0 K 2023-08-07 08:53:32,600 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/.tmp/cf2/de611d4e2cf544928b1f9d901618602a as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/de611d4e2cf544928b1f9d901618602a 2023-08-07 08:53:32,605 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/de611d4e2cf544928b1f9d901618602a, entries=10, sequenceid=58, filesize=5.0 K 2023-08-07 08:53:32,606 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for fd83112e7ec055aba3e5a554dab2841e in 39ms, sequenceid=58, compaction requested=true 2023-08-07 08:53:32,606 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fd83112e7ec055aba3e5a554dab2841e: 2023-08-07 08:53:32,606 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. 2023-08-07 08:53:32,606 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:53:32,606 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:53:32,606 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,606 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-07 08:53:32,606 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-07 08:53:32,609 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,609 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:32,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:32,609 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,609 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:32,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:32,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:32,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-07 08:53:32,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:32,611 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-08-07 08:53:32,611 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@72e01b03[Count = 0] remaining members to acquire global barrier 2023-08-07 08:53:32,611 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-08-07 08:53:32,611 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,613 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,613 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,613 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,613 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,613 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-07 08:53:32,613 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,613 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,613 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-07 08:53:32,613 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-07 08:53:32,613 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-07 08:53:32,613 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,613 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:53:32,613 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:53:32,613 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:53:32,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,613 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-07 08:53:32,613 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-07 08:53:32,613 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:53:32,615 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-07 08:53:32,615 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:32,615 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-07 08:53:32,619 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-07 08:53:32,619 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,619 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:32,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:32,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:32,619 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-07 08:53:32,619 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-07 08:53:32,619 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:32,619 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-07 08:53:32,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:32,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:32,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-07 08:53:32,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:32,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-07 08:53:32,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,34559,1691398396467': 2023-08-07 08:53:32,622 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-08-07 08:53:32,622 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-08-07 08:53:32,622 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:53:32,622 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:53:32,622 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-08-07 08:53:32,622 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:53:32,624 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,624 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,624 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,624 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,624 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:32,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,624 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:32,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,624 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:32,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,625 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:32,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:32,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:32,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:32,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:32,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:32,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:32,625 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:32,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:32,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,626 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-07 08:53:32,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:32,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-07 08:53:32,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,628 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:32,628 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-07 08:53:32,628 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,631 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:32,631 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:32,631 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:32,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:32,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:32,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:32,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:32,631 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:32,631 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:32,632 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:32,631 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-08-07 08:53:32,632 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:53:32,632 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-08-07 08:53:32,632 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:53:32,631 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:32,631 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:32,632 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:32,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:32,632 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:32,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:32,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:32,632 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:32,633 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:32,633 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,633 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,633 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,633 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:32,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:32,633 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,633 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,732 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:53:32,733 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:53:32,749 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:53:32,751 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-08-07 08:53:32,751 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-08-07 08:53:32,751 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-08-07 08:53:32,751 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:32,751 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-08-07 08:53:32,751 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-08-07 08:53:32,752 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,752 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:32,801 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:32,801 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:32,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:32,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:32,801 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:32,801 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:32,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:32,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:32,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:32,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:32,802 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,802 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:53:32,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:32,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:32,802 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,802 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,802 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:32,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:32,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:32,803 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-07 08:53:32,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:32,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:32,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:32,803 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-07 08:53:32,803 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-07 08:53:32,803 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-07 08:53:32,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:32,803 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-07 08:53:32,803 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:32,803 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-07 08:53:32,803 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-07 08:53:32,803 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-07 08:53:32,803 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-07 08:53:32,803 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-07 08:53:32,803 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-07 08:53:32,803 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-07 08:53:32,803 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:32,803 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-07 08:53:32,803 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-07 08:53:32,804 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-07 08:53:32,804 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-07 08:53:32,804 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:53:32,804 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. 2023-08-07 08:53:32,804 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:53:32,804 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. started... 2023-08-07 08:53:32,804 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:32,804 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-07 08:53:32,804 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-07 08:53:32,804 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-07 08:53:32,804 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fd83112e7ec055aba3e5a554dab2841e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:53:32,815 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/.tmp/cf1/143645e1480743c09a8dbb82da50c40b 2023-08-07 08:53:32,828 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/.tmp/cf2/ec5dd4ae9931452288493d04a2a1502d 2023-08-07 08:53:32,834 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/.tmp/cf1/143645e1480743c09a8dbb82da50c40b as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/143645e1480743c09a8dbb82da50c40b 2023-08-07 08:53:32,839 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/143645e1480743c09a8dbb82da50c40b, entries=10, sequenceid=71, filesize=5.0 K 2023-08-07 08:53:32,840 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/.tmp/cf2/ec5dd4ae9931452288493d04a2a1502d as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/ec5dd4ae9931452288493d04a2a1502d 2023-08-07 08:53:32,845 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/ec5dd4ae9931452288493d04a2a1502d, entries=10, sequenceid=71, filesize=5.0 K 2023-08-07 08:53:32,846 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for fd83112e7ec055aba3e5a554dab2841e in 42ms, sequenceid=71, compaction requested=true 2023-08-07 08:53:32,846 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fd83112e7ec055aba3e5a554dab2841e: 2023-08-07 08:53:32,846 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. 2023-08-07 08:53:32,846 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:53:32,846 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:53:32,846 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,846 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-07 08:53:32,846 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-07 08:53:32,862 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,894 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,894 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,894 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,894 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,895 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:32,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:32,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:32,895 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,895 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:32,895 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,895 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:32,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:32,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:32,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-07 08:53:32,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:32,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:32,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:32,897 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-08-07 08:53:32,897 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4920cd3[Count = 0] remaining members to acquire global barrier 2023-08-07 08:53:32,897 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-08-07 08:53:32,897 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,939 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,939 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,939 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,940 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:32,940 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:53:32,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,940 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-07 08:53:32,940 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-07 08:53:32,940 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-07 08:53:32,940 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:53:32,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:32,940 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-07 08:53:32,940 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:53:32,940 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-07 08:53:32,940 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-07 08:53:32,940 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:53:32,993 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-07 08:53:32,993 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:32,993 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-07 08:53:33,032 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:33,032 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-07 08:53:33,032 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-07 08:53:33,032 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:33,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:33,032 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-07 08:53:33,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:33,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:33,032 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:33,032 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-07 08:53:33,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:33,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:33,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-07 08:53:33,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:33,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:33,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:33,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:33,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-07 08:53:33,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:33,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:33,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:33,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,34559,1691398396467': 2023-08-07 08:53:33,036 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-08-07 08:53:33,036 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-08-07 08:53:33,036 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:53:33,036 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:53:33,036 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-08-07 08:53:33,036 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:53:33,062 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:33,062 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:33,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:33,063 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:33,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:33,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:33,063 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:33,062 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:33,062 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:33,063 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:33,063 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:33,063 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:33,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:33,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:33,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:33,063 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:33,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:33,063 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:33,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-07 08:53:33,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:33,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:33,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:33,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:33,064 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:33,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:33,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:33,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:33,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:33,064 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:33,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:33,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-07 08:53:33,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:33,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:33,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:33,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:33,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:33,066 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:33,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-07 08:53:33,066 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:33,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:33,067 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:33,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:33,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:33,167 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:33,167 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:33,167 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:33,167 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:33,167 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:33,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:33,167 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:33,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:33,167 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:33,167 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:33,168 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:33,168 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-07 08:53:33,168 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:33,168 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:33,168 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:33,168 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:33,168 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-07 08:53:33,168 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-07 08:53:33,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:33,167 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-08-07 08:53:33,168 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:53:33,167 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:33,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:33,167 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:33,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:33,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:33,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:33,168 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-08-07 08:53:33,168 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:53:33,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:33,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:33,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:33,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:33,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:33,269 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:53:33,270 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:53:33,529 WARN [Listener at localhost/33561] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-08-07 08:53:33,531 DEBUG [Listener at localhost/33561] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:33,532 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(1029): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:33,533 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:33,533 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:33,533 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:33,534 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:53:33,534 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:33,534 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:53:33,534 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:53:33,782 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:33,782 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:53:33,782 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:53:33,783 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:53:33,783 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-08-07 08:53:33,783 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:53:33,783 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:53:33,783 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:53:33,783 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:53:33,784 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:53:33,785 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-08-07 08:53:33,785 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:53:33,786 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:53:33,786 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:53:33,786 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:53:33,787 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:33,787 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:53:33,787 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:33,787 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:33,788 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:33,788 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:33,788 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:33,788 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:33,789 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:53:33,789 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:33,789 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:53:33,789 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:33,789 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:33,789 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:33,789 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-08-07 08:53:33,790 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:53:33,791 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:53:33,791 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398443791 2023-08-07 08:53:33,792 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:53:33,793 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:33,795 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:33,795 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=fd83112e7ec055aba3e5a554dab2841e, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34559,1691398396467, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467, openSeqNum=2 2023-08-07 08:53:33,795 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:53:33,796 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398473796 2023-08-07 08:53:33,796 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398533796 2023-08-07 08:53:33,796 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-07 08:53:33,796 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:53:33,798 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:53:33,798 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:33,798 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:53:33,798 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:53:34,046 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:34,046 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:53:34,046 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:53:34,046 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:53:34,047 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 08:53:34,047 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:53:34,048 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:53:34,048 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:53:34,048 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:53:34,048 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:53:34,050 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-08-07 08:53:34,050 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:53:34,051 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:53:34,051 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:53:34,051 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:53:34,052 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:34,052 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:53:34,052 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:34,053 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:34,053 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:34,054 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:34,054 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:34,054 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:34,054 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:53:34,055 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:34,055 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:53:34,055 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:34,055 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:34,055 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:34,056 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-08-07 08:53:34,056 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:53:34,057 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:53:34,057 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398444057 2023-08-07 08:53:34,059 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:34,059 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:53:34,061 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:34,061 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=fd83112e7ec055aba3e5a554dab2841e, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34559,1691398396467, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467, openSeqNum=2 2023-08-07 08:53:34,061 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:53:34,061 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398474061 2023-08-07 08:53:34,062 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398534062 2023-08-07 08:53:34,062 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-08-07 08:53:34,062 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:53:34,064 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN}] 2023-08-07 08:53:34,064 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:53:34,064 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:34,064 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:53:34,064 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:53:34,312 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:34,312 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:53:34,312 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:53:34,312 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:53:34,313 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 08:53:34,313 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:53:34,313 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:53:34,313 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:53:34,314 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:53:34,314 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:53:34,315 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-08-07 08:53:34,316 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:53:34,316 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:53:34,316 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:53:34,316 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:53:34,317 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:34,317 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:53:34,317 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:34,317 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:34,318 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:34,318 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:34,318 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:34,319 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:34,319 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:53:34,319 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:34,319 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:53:34,320 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:34,320 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:34,320 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:34,320 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-08-07 08:53:34,321 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:53:34,321 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:53:34,321 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398444321 2023-08-07 08:53:34,321 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:53:34,323 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:34,325 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:34,325 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=fd83112e7ec055aba3e5a554dab2841e, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34559,1691398396467, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467, openSeqNum=2 2023-08-07 08:53:34,325 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:53:34,325 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398474325 2023-08-07 08:53:34,325 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398534325 2023-08-07 08:53:34,325 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-07 08:53:34,326 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN}] 2023-08-07 08:53:34,326 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:53:34,328 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN 2023-08-07 08:53:34,329 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=fd83112e7ec055aba3e5a554dab2841e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:34,329 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691398414329"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398414329"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398414329"}]},"ts":"1691398414329"} 2023-08-07 08:53:34,331 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure fd83112e7ec055aba3e5a554dab2841e, server=jenkins-hbase4.apache.org,34559,1691398396467}] 2023-08-07 08:53:34,331 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:53:34,331 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN 2023-08-07 08:53:34,331 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:53:34,331 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:53:34,576 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:34,576 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:53:34,576 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:53:34,576 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:53:34,576 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 08:53:34,576 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:53:34,576 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:53:34,576 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:53:34,577 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:53:34,577 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:53:34,578 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-08-07 08:53:34,578 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-08-07 08:53:34,579 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:53:34,579 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:53:34,579 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:53:34,579 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:53:34,580 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:34,580 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:53:34,580 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:34,580 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:34,581 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:34,581 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:34,581 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:34,581 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:34,581 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:53:34,582 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:34,582 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN 2023-08-07 08:53:34,582 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:53:34,582 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:34,582 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:34,583 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:34,583 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:53:34,583 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN 2023-08-07 08:53:34,583 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-08-07 08:53:34,583 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:53:34,584 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:53:34,584 INFO [Listener at localhost/33561] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e to restore RIT 2023-08-07 08:53:34,584 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398444584 2023-08-07 08:53:34,585 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:53:34,585 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN 2023-08-07 08:53:34,587 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:34,588 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=fd83112e7ec055aba3e5a554dab2841e, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,34559,1691398396467, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467, openSeqNum=2 2023-08-07 08:53:34,588 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:53:34,588 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398474588 2023-08-07 08:53:34,588 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398534588 2023-08-07 08:53:34,588 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-08-07 08:53:34,588 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=fd83112e7ec055aba3e5a554dab2841e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:34,588 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691398414588"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398414588"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398414588"}]},"ts":"1691398414588"} 2023-08-07 08:53:34,590 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure fd83112e7ec055aba3e5a554dab2841e, server=jenkins-hbase4.apache.org,34559,1691398396467}] 2023-08-07 08:53:34,590 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:53:34,743 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:34,743 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing fd83112e7ec055aba3e5a554dab2841e, disabling compactions & flushes 2023-08-07 08:53:34,743 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. 2023-08-07 08:53:34,743 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. 2023-08-07 08:53:34,743 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. after waiting 0 ms 2023-08-07 08:53:34,743 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. 2023-08-07 08:53:34,743 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing fd83112e7ec055aba3e5a554dab2841e 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-08-07 08:53:34,756 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/.tmp/cf1/42ed52d5acc545a89cbdfc54492d824e 2023-08-07 08:53:34,778 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/.tmp/cf2/e328868cd54b41f393a38958421a6b19 2023-08-07 08:53:34,784 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/.tmp/cf1/42ed52d5acc545a89cbdfc54492d824e as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/42ed52d5acc545a89cbdfc54492d824e 2023-08-07 08:53:34,789 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/42ed52d5acc545a89cbdfc54492d824e, entries=8, sequenceid=82, filesize=5.0 K 2023-08-07 08:53:34,790 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/.tmp/cf2/e328868cd54b41f393a38958421a6b19 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/e328868cd54b41f393a38958421a6b19 2023-08-07 08:53:34,795 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/e328868cd54b41f393a38958421a6b19, entries=8, sequenceid=82, filesize=5.0 K 2023-08-07 08:53:34,796 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for fd83112e7ec055aba3e5a554dab2841e in 53ms, sequenceid=82, compaction requested=true 2023-08-07 08:53:34,802 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-08-07 08:53:34,803 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-07 08:53:34,803 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. 2023-08-07 08:53:34,803 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for fd83112e7ec055aba3e5a554dab2841e: 2023-08-07 08:53:34,806 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:34,806 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=fd83112e7ec055aba3e5a554dab2841e, regionState=CLOSED 2023-08-07 08:53:34,806 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691398414806"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398414806"}]},"ts":"1691398414806"} 2023-08-07 08:53:34,808 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:53:34,808 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure fd83112e7ec055aba3e5a554dab2841e, server=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:34,808 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:53:34,808 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure fd83112e7ec055aba3e5a554dab2841e, server=jenkins-hbase4.apache.org,34559,1691398396467 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:53:34,838 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:34,838 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:53:34,838 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:53:34,838 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:53:34,839 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 08:53:34,839 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:53:34,839 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:53:34,839 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:53:34,839 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:53:34,840 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:53:34,841 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-08-07 08:53:34,841 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-08-07 08:53:34,841 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-08-07 08:53:34,841 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:53:34,841 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:53:34,842 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:53:34,842 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:53:34,842 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:34,842 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:53:34,843 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:34,843 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:34,843 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:34,844 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:34,844 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:34,844 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:34,844 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:53:34,845 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:34,845 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN 2023-08-07 08:53:34,845 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure fd83112e7ec055aba3e5a554dab2841e, server=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:34,845 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:53:34,846 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:34,846 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:34,846 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:34,846 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:53:34,846 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN 2023-08-07 08:53:34,846 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure fd83112e7ec055aba3e5a554dab2841e, server=jenkins-hbase4.apache.org,34559,1691398396467 held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:53:34,846 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-08-07 08:53:34,847 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:53:34,848 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:53:34,848 INFO [Listener at localhost/33561] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e to restore RIT 2023-08-07 08:53:34,848 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398444848 2023-08-07 08:53:34,849 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure fd83112e7ec055aba3e5a554dab2841e, server=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:34,849 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:53:34,852 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:34,852 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=fd83112e7ec055aba3e5a554dab2841e, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34559,1691398396467, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467, openSeqNum=2 2023-08-07 08:53:34,852 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:53:34,852 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398474852 2023-08-07 08:53:34,852 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398534852 2023-08-07 08:53:34,853 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-07 08:53:34,853 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=fd83112e7ec055aba3e5a554dab2841e, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:34,853 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691398414853"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398414853"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398414853"}]},"ts":"1691398414853"} 2023-08-07 08:53:34,854 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:53:34,857 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-08-07 08:53:34,857 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure fd83112e7ec055aba3e5a554dab2841e, server=jenkins-hbase4.apache.org,34559,1691398396467 in 264 msec 2023-08-07 08:53:34,858 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:53:34,858 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN 2023-08-07 08:53:34,858 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:53:34,858 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:53:35,103 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:35,103 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:53:35,103 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:53:35,103 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:53:35,106 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 08:53:35,106 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:53:35,106 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:53:35,106 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:53:35,106 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:53:35,107 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:53:35,108 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-08-07 08:53:35,108 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-08-07 08:53:35,108 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-07 08:53:35,109 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:53:35,109 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:53:35,109 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:53:35,110 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:53:35,110 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:35,110 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:53:35,111 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:35,111 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:35,111 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:35,112 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:35,112 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:35,112 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:35,112 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:53:35,113 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:35,113 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN 2023-08-07 08:53:35,113 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure fd83112e7ec055aba3e5a554dab2841e, server=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:35,114 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:53:35,114 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:35,114 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:35,114 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:35,114 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:53:35,114 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure fd83112e7ec055aba3e5a554dab2841e, server=jenkins-hbase4.apache.org,34559,1691398396467 didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:53:35,114 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-08-07 08:53:35,115 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:53:35,115 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:53:35,116 INFO [Listener at localhost/33561] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e to restore RIT 2023-08-07 08:53:35,116 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398445116 2023-08-07 08:53:35,117 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:53:35,117 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN 2023-08-07 08:53:35,119 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:35,119 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=fd83112e7ec055aba3e5a554dab2841e, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34559,1691398396467, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467, openSeqNum=2 2023-08-07 08:53:35,120 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:53:35,120 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398475120 2023-08-07 08:53:35,120 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398535120 2023-08-07 08:53:35,120 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-08-07 08:53:35,120 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN 2023-08-07 08:53:35,122 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:53:35,125 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-08-07 08:53:35,125 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN in 796 msec 2023-08-07 08:53:35,128 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:53:35,128 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:35,128 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:53:35,128 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:53:35,370 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:35,370 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:53:35,370 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:53:35,370 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:53:35,370 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 08:53:35,371 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:53:35,371 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:53:35,371 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:53:35,371 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:53:35,371 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:53:35,372 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-08-07 08:53:35,372 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-07 08:53:35,372 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-07 08:53:35,373 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:53:35,373 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:53:35,373 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:53:35,373 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:53:35,374 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:35,374 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:53:35,374 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:35,375 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:35,375 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:35,375 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:35,375 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:35,376 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:35,376 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:53:35,376 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:35,377 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN 2023-08-07 08:53:35,377 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure fd83112e7ec055aba3e5a554dab2841e, server=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:35,377 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:53:35,377 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:53:35,377 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure fd83112e7ec055aba3e5a554dab2841e, server=jenkins-hbase4.apache.org,34559,1691398396467 didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:53:35,377 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-08-07 08:53:35,378 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:53:35,378 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:53:35,379 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398445378 2023-08-07 08:53:35,379 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:53:35,380 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:35,383 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:35,383 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=fd83112e7ec055aba3e5a554dab2841e, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34559,1691398396467, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467, openSeqNum=2 2023-08-07 08:53:35,383 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:53:35,383 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398475383 2023-08-07 08:53:35,383 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398535383 2023-08-07 08:53:35,383 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-08-07 08:53:35,384 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:35,384 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:35,384 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:35,389 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:53:35,400 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=fd83112e7ec055aba3e5a554dab2841e, threads=14 2023-08-07 08:53:35,401 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/1ca97b2be41f490591a5050a91d41fa9 for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:35,401 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/acf834d7b8cb41aea32f6e17233515cf for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:35,401 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/da3a5ec77c6c42c4a4be4511e65db6bc for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:35,402 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/de611d4e2cf544928b1f9d901618602a for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:35,402 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/e328868cd54b41f393a38958421a6b19 for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:35,408 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/ec5dd4ae9931452288493d04a2a1502d for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:35,409 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/f5df38fe79b345d783c503b0e7ebbd90 for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:35,410 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/143645e1480743c09a8dbb82da50c40b for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:35,410 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/2489c9607b2645d5845bd72a39dbb77b for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:35,410 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/42ed52d5acc545a89cbdfc54492d824e for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:35,411 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/6b8a137fb9644c108395222874065f46 for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:35,411 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/77fb8078d2014f87887d92cddcb5d75e for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:35,411 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/a2e4aab8fc0f4f01bed8c08d5e21e5ad for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:35,417 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/ac88c845249d46e49f0532262366d399 for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:35,658 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/acf834d7b8cb41aea32f6e17233515cf for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:35,671 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/2489c9607b2645d5845bd72a39dbb77b for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:35,673 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/e328868cd54b41f393a38958421a6b19 for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:35,673 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/a2e4aab8fc0f4f01bed8c08d5e21e5ad for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:35,702 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/6b8a137fb9644c108395222874065f46 for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:35,713 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-08-07 08:53:35,717 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/77fb8078d2014f87887d92cddcb5d75e for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:35,736 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/ac88c845249d46e49f0532262366d399 for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:35,743 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/42ed52d5acc545a89cbdfc54492d824e for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:35,753 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/ec5dd4ae9931452288493d04a2a1502d for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:35,754 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/143645e1480743c09a8dbb82da50c40b for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:35,756 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/f5df38fe79b345d783c503b0e7ebbd90 for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:35,757 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/de611d4e2cf544928b1f9d901618602a for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:35,758 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/1ca97b2be41f490591a5050a91d41fa9 for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:36,155 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/da3a5ec77c6c42c4a4be4511e65db6bc for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:36,155 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region fd83112e7ec055aba3e5a554dab2841e Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-08-07 08:53:36,203 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:53:36,203 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:36,203 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:53:36,204 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:53:36,384 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:36,384 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:53:36,384 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:53:36,384 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:53:36,385 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 08:53:36,386 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:53:36,386 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:53:36,386 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:53:36,386 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:53:36,386 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:53:36,387 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-08-07 08:53:36,387 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-07 08:53:36,388 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-07 08:53:36,388 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:53:36,388 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:53:36,388 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:53:36,388 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:53:36,389 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:36,389 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:53:36,389 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:36,389 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:36,389 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:36,390 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:36,390 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:36,390 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:36,390 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:53:36,391 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:36,391 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN 2023-08-07 08:53:36,391 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure fd83112e7ec055aba3e5a554dab2841e, server=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:36,391 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:53:36,391 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:36,391 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:36,391 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:36,391 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:53:36,391 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure fd83112e7ec055aba3e5a554dab2841e, server=jenkins-hbase4.apache.org,34559,1691398396467 didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:53:36,391 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-08-07 08:53:36,392 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:53:36,392 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:53:36,393 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398446393 2023-08-07 08:53:36,394 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:36,394 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:53:36,396 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:36,397 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=fd83112e7ec055aba3e5a554dab2841e, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34559,1691398396467, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467, openSeqNum=2 2023-08-07 08:53:36,397 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:53:36,397 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398476397 2023-08-07 08:53:36,397 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398536397 2023-08-07 08:53:36,397 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-08-07 08:53:36,407 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=fd83112e7ec055aba3e5a554dab2841e, threads=14 2023-08-07 08:53:36,409 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/1ca97b2be41f490591a5050a91d41fa9 for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:36,409 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/acf834d7b8cb41aea32f6e17233515cf for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:36,409 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/da3a5ec77c6c42c4a4be4511e65db6bc for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:36,410 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/e328868cd54b41f393a38958421a6b19 for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:36,410 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/de611d4e2cf544928b1f9d901618602a for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:36,410 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/ec5dd4ae9931452288493d04a2a1502d for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:36,411 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/f5df38fe79b345d783c503b0e7ebbd90 for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:36,411 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/143645e1480743c09a8dbb82da50c40b for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:36,413 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/2489c9607b2645d5845bd72a39dbb77b for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:36,414 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/42ed52d5acc545a89cbdfc54492d824e for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:36,414 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/6b8a137fb9644c108395222874065f46 for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:36,414 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/77fb8078d2014f87887d92cddcb5d75e for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:36,415 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/a2e4aab8fc0f4f01bed8c08d5e21e5ad for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:36,415 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/ac88c845249d46e49f0532262366d399 for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:36,585 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/ac88c845249d46e49f0532262366d399 for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:36,586 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/acf834d7b8cb41aea32f6e17233515cf for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:36,597 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/77fb8078d2014f87887d92cddcb5d75e for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:36,604 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/de611d4e2cf544928b1f9d901618602a for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:36,615 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/ec5dd4ae9931452288493d04a2a1502d for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:36,616 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/6b8a137fb9644c108395222874065f46 for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:36,635 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/143645e1480743c09a8dbb82da50c40b for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:36,644 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/42ed52d5acc545a89cbdfc54492d824e for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:36,650 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/a2e4aab8fc0f4f01bed8c08d5e21e5ad for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:36,655 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/da3a5ec77c6c42c4a4be4511e65db6bc for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:36,663 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/1ca97b2be41f490591a5050a91d41fa9 for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:36,663 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/2489c9607b2645d5845bd72a39dbb77b for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:36,663 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/e328868cd54b41f393a38958421a6b19 for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:36,666 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/f5df38fe79b345d783c503b0e7ebbd90 for region: fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:36,666 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region fd83112e7ec055aba3e5a554dab2841e Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-08-07 08:53:36,711 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:53:36,718 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/48c2ad6fc762203958def153b6e206ef/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-08-07 08:53:36,720 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/1eb65ab1dd6aaf4ea2b55584fc60246a/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-08-07 08:53:36,720 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:53:36,720 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:36,720 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:53:36,720 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:53:36,897 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:36,897 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:53:36,897 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:53:36,897 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:53:36,898 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 08:53:36,898 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:53:36,898 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:53:36,898 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:53:36,898 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:53:36,899 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:53:36,900 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-08-07 08:53:36,900 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-07 08:53:36,900 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-07 08:53:36,900 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:53:36,901 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:53:36,901 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:53:36,901 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:53:36,902 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:36,902 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:53:36,902 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:36,902 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:36,903 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:36,903 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:36,903 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:36,903 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:36,903 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:53:36,904 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:36,904 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN 2023-08-07 08:53:36,904 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure fd83112e7ec055aba3e5a554dab2841e, server=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:36,907 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:53:36,907 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:36,907 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:36,907 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:36,907 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:53:36,908 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure fd83112e7ec055aba3e5a554dab2841e, server=jenkins-hbase4.apache.org,34559,1691398396467 didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:53:36,908 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-08-07 08:53:36,908 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:53:36,909 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:53:36,909 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398446909 2023-08-07 08:53:36,910 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:53:36,911 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:36,914 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:36,914 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=fd83112e7ec055aba3e5a554dab2841e, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34559,1691398396467, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467, openSeqNum=2 2023-08-07 08:53:36,914 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:53:36,914 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398476914 2023-08-07 08:53:36,914 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398536914 2023-08-07 08:53:36,914 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-08-07 08:53:36,918 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:53:36,921 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:53:36,921 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:36,921 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:53:36,921 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:53:37,165 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:37,165 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:53:37,165 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:53:37,166 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:53:37,166 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 08:53:37,166 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:53:37,166 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:53:37,166 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:53:37,166 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:53:37,167 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:53:37,168 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-08-07 08:53:37,168 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-07 08:53:37,168 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-07 08:53:37,168 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:53:37,169 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:53:37,169 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:53:37,169 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:53:37,169 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:37,170 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:53:37,170 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:37,170 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:37,170 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:37,170 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:37,170 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:37,171 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:37,171 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:53:37,171 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:37,171 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN 2023-08-07 08:53:37,171 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure fd83112e7ec055aba3e5a554dab2841e, server=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:37,172 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:53:37,172 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:37,172 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:37,172 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:37,172 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:53:37,172 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure fd83112e7ec055aba3e5a554dab2841e, server=jenkins-hbase4.apache.org,34559,1691398396467 didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:53:37,172 DEBUG [Listener at localhost/33561] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:37,172 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-08-07 08:53:37,173 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:53:37,173 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:53:37,174 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398447174 2023-08-07 08:53:37,175 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:53:37,177 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:37,180 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:37,180 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=fd83112e7ec055aba3e5a554dab2841e, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34559,1691398396467, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467, openSeqNum=2 2023-08-07 08:53:37,180 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:53:37,180 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398477180 2023-08-07 08:53:37,180 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398537180 2023-08-07 08:53:37,180 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-08-07 08:53:37,181 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:53:37,194 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:53:37,194 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-08-07 08:53:37,194 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:53:37,194 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:53:37,431 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:37,432 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:53:37,432 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:53:37,432 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:53:37,432 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 08:53:37,432 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:53:37,432 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:53:37,432 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:53:37,432 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:53:37,433 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:53:37,434 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-08-07 08:53:37,434 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-07 08:53:37,434 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-07 08:53:37,435 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:53:37,435 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:53:37,435 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:53:37,435 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:53:37,435 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:37,436 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:53:37,436 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:37,436 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:37,436 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:37,436 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:37,437 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:37,437 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:37,437 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:53:37,438 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:37,438 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN 2023-08-07 08:53:37,438 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure fd83112e7ec055aba3e5a554dab2841e, server=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:37,439 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:53:37,439 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:37,439 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:37,439 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:37,439 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:53:37,439 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure fd83112e7ec055aba3e5a554dab2841e, server=jenkins-hbase4.apache.org,34559,1691398396467 didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:53:37,439 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-08-07 08:53:37,441 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:53:37,441 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:53:37,441 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398447441 2023-08-07 08:53:37,444 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:53:37,445 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:37,447 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:37,447 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=fd83112e7ec055aba3e5a554dab2841e, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34559,1691398396467, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467, openSeqNum=2 2023-08-07 08:53:37,447 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:53:37,447 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398477447 2023-08-07 08:53:37,447 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398537447 2023-08-07 08:53:37,447 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-07 08:53:37,447 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:53:37,449 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:53:37,449 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-08-07 08:53:37,449 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:53:37,449 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:53:37,698 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:37,698 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:53:37,698 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:53:37,698 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:53:37,699 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 08:53:37,699 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:53:37,699 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:53:37,699 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:53:37,699 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:53:37,700 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:53:37,700 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-08-07 08:53:37,701 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-07 08:53:37,701 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-07 08:53:37,701 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:53:37,702 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:53:37,702 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:53:37,702 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:53:37,702 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:37,702 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:53:37,702 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:37,703 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:37,703 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:37,703 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:37,703 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:37,703 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:37,704 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:53:37,704 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:37,704 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN 2023-08-07 08:53:37,704 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure fd83112e7ec055aba3e5a554dab2841e, server=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:37,705 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:53:37,705 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:37,705 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:37,705 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:37,705 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:53:37,705 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure fd83112e7ec055aba3e5a554dab2841e, server=jenkins-hbase4.apache.org,34559,1691398396467 didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:53:37,705 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-08-07 08:53:37,706 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:53:37,706 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:53:37,706 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398447706 2023-08-07 08:53:37,708 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:53:37,708 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:37,710 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:37,711 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=fd83112e7ec055aba3e5a554dab2841e, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34559,1691398396467, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467, openSeqNum=2 2023-08-07 08:53:37,711 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:53:37,711 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398477711 2023-08-07 08:53:37,711 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398537711 2023-08-07 08:53:37,711 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-07 08:53:37,711 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:53:37,713 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:53:37,713 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-08-07 08:53:37,713 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:53:37,713 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:53:37,961 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:37,961 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:53:37,962 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:53:37,962 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:53:37,963 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 08:53:37,963 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:53:37,963 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:53:37,963 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:53:37,963 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:53:37,964 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:53:37,965 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-08-07 08:53:37,966 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-07 08:53:37,966 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-07 08:53:37,966 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:53:37,966 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:53:37,967 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:53:37,967 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:53:37,967 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:37,968 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:53:37,968 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:37,968 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:37,968 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:37,969 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:37,969 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:37,969 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:37,969 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:53:37,970 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:37,970 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN 2023-08-07 08:53:37,970 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure fd83112e7ec055aba3e5a554dab2841e, server=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:37,970 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:53:37,970 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:37,971 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:37,971 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:37,971 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:53:37,971 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure fd83112e7ec055aba3e5a554dab2841e, server=jenkins-hbase4.apache.org,34559,1691398396467 didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:53:37,971 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-08-07 08:53:37,972 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:53:37,972 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:53:37,973 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398447973 2023-08-07 08:53:37,975 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:37,975 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:53:37,977 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:37,978 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=fd83112e7ec055aba3e5a554dab2841e, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34559,1691398396467, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467, openSeqNum=2 2023-08-07 08:53:37,978 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:53:37,978 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398477978 2023-08-07 08:53:37,978 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398537978 2023-08-07 08:53:37,978 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-08-07 08:53:37,978 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:53:37,982 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1054): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN] 2023-08-07 08:53:37,982 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:53:37,982 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-08-07 08:53:37,982 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:53:37,982 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:53:38,228 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:38,229 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:53:38,229 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:53:38,229 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:53:38,230 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 08:53:38,230 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:53:38,230 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:53:38,230 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:53:38,230 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:53:38,231 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:53:38,232 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-08-07 08:53:38,233 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-07 08:53:38,233 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-07 08:53:38,233 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-08-07 08:53:38,233 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:53:38,234 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:53:38,234 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:53:38,234 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:53:38,234 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:38,235 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:53:38,235 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:38,235 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:38,236 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:38,236 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:38,236 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:38,236 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:38,237 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:53:38,238 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:38,238 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN 2023-08-07 08:53:38,238 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure fd83112e7ec055aba3e5a554dab2841e, server=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:38,238 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN 2023-08-07 08:53:38,238 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:53:38,238 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:38,239 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:38,239 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:38,239 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:53:38,239 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure fd83112e7ec055aba3e5a554dab2841e, server=jenkins-hbase4.apache.org,34559,1691398396467 didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:53:38,239 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:53:38,239 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-08-07 08:53:38,240 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:53:38,240 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:53:38,241 INFO [Listener at localhost/33561] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e to restore RIT 2023-08-07 08:53:38,241 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398448241 2023-08-07 08:53:38,243 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:38,243 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:53:38,244 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN 2023-08-07 08:53:38,246 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:38,246 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=fd83112e7ec055aba3e5a554dab2841e, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34559,1691398396467, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467, openSeqNum=2 2023-08-07 08:53:38,247 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:53:38,247 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398478247 2023-08-07 08:53:38,247 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398538247 2023-08-07 08:53:38,247 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-08-07 08:53:38,247 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:53:38,252 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:53:38,252 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-08-07 08:53:38,252 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:53:38,252 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:53:38,497 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:38,497 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:53:38,497 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:53:38,497 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:53:38,499 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 08:53:38,499 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:53:38,499 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:53:38,499 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:53:38,499 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:53:38,500 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:53:38,501 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-08-07 08:53:38,501 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-08-07 08:53:38,501 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:53:38,501 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:53:38,502 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:53:38,502 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:53:38,502 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:38,503 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:53:38,503 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:38,503 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:38,503 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:38,504 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:38,504 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:38,504 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:38,504 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:53:38,505 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:38,505 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN 2023-08-07 08:53:38,505 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:53:38,505 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:38,505 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:38,505 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:38,506 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:53:38,506 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-08-07 08:53:38,506 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:53:38,507 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:53:38,507 INFO [Listener at localhost/33561] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e to restore RIT 2023-08-07 08:53:38,507 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398448507 2023-08-07 08:53:38,509 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:38,509 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:53:38,510 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN 2023-08-07 08:53:38,512 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:38,512 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=fd83112e7ec055aba3e5a554dab2841e, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34559,1691398396467, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467, openSeqNum=2 2023-08-07 08:53:38,513 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:53:38,513 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398478513 2023-08-07 08:53:38,513 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398538513 2023-08-07 08:53:38,513 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-07 08:53:38,513 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:53:38,515 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:53:38,515 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-08-07 08:53:38,515 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:53:38,515 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:53:38,763 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:38,763 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:53:38,763 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:53:38,764 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:53:38,765 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 08:53:38,765 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:53:38,765 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:53:38,765 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:53:38,765 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:53:38,766 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:53:38,767 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-08-07 08:53:38,767 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-08-07 08:53:38,767 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:53:38,768 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:53:38,768 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:53:38,768 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:53:38,769 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:38,769 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:53:38,769 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:38,770 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:38,770 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:38,770 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:38,770 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:38,771 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:38,771 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:53:38,771 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:38,772 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN 2023-08-07 08:53:38,772 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:53:38,772 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:38,772 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:38,772 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:38,772 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:53:38,772 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-08-07 08:53:38,773 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:53:38,774 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:53:38,774 INFO [Listener at localhost/33561] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e to restore RIT 2023-08-07 08:53:38,774 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398448774 2023-08-07 08:53:38,775 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:53:38,776 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:38,777 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN 2023-08-07 08:53:38,779 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:38,779 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=fd83112e7ec055aba3e5a554dab2841e, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34559,1691398396467, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467, openSeqNum=2 2023-08-07 08:53:38,779 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:53:38,779 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398478779 2023-08-07 08:53:38,779 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398538779 2023-08-07 08:53:38,779 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-08-07 08:53:38,779 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:53:38,781 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:53:38,781 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-08-07 08:53:38,781 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:53:38,781 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:53:39,029 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:39,029 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:53:39,030 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:53:39,030 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:53:39,031 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 08:53:39,031 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:53:39,031 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:53:39,031 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:53:39,031 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:53:39,032 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:53:39,032 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-08-07 08:53:39,033 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-08-07 08:53:39,033 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:53:39,033 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:53:39,033 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:53:39,033 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:53:39,034 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:39,034 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:53:39,034 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:39,034 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:39,034 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:39,035 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:39,035 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:39,035 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:39,035 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:53:39,035 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:39,035 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN 2023-08-07 08:53:39,035 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:53:39,036 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:39,036 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:39,036 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:39,036 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:53:39,036 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-08-07 08:53:39,036 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:53:39,037 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:53:39,037 INFO [Listener at localhost/33561] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e to restore RIT 2023-08-07 08:53:39,037 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398449037 2023-08-07 08:53:39,039 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:53:39,039 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:39,041 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN 2023-08-07 08:53:39,043 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:39,043 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=fd83112e7ec055aba3e5a554dab2841e, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34559,1691398396467, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467, openSeqNum=2 2023-08-07 08:53:39,043 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:53:39,043 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398479043 2023-08-07 08:53:39,044 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398539044 2023-08-07 08:53:39,044 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 6 msec 2023-08-07 08:53:39,044 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:53:39,046 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a exec-time=5.5130 sec 2023-08-07 08:53:39,046 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN 2023-08-07 08:53:39,047 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,34559,1691398396467; forceNewPlan=false, retain=true 2023-08-07 08:53:39,197 INFO [jenkins-hbase4:33357] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-07 08:53:39,199 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=fd83112e7ec055aba3e5a554dab2841e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:39,199 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691398419198"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398419198"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398419198"}]},"ts":"1691398419198"} 2023-08-07 08:53:39,201 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure fd83112e7ec055aba3e5a554dab2841e, server=jenkins-hbase4.apache.org,34559,1691398396467}] 2023-08-07 08:53:39,201 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:53:39,201 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN 2023-08-07 08:53:39,201 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:53:39,201 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:53:39,294 WARN [Listener at localhost/33561] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-07 08:53:39,294 WARN [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-08-07 08:53:39,294 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:53:39,294 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:53:39,294 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:53:39,296 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 08:53:39,297 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:53:39,297 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:53:39,297 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:53:39,297 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:53:39,298 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:53:39,299 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-08-07 08:53:39,299 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:53:39,299 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:53:39,299 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:53:39,299 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:53:39,300 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:39,300 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:53:39,300 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:39,300 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:53:39,300 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:39,300 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:53:39,301 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:39,301 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-07 08:53:39,301 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:53:39,302 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:53:39,302 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN 2023-08-07 08:53:39,302 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:53:39,302 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN 2023-08-07 08:53:39,302 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-08-07 08:53:39,303 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:53:39,303 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:53:39,303 INFO [Listener at localhost/33561] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e to restore RIT 2023-08-07 08:53:39,308 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398449308 2023-08-07 08:53:39,308 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:53:39,309 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN 2023-08-07 08:53:39,310 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:53:39,311 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=fd83112e7ec055aba3e5a554dab2841e, regionState=OPENING, lastHost=jenkins-hbase4.apache.org,34559,1691398396467, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467, openSeqNum=2 2023-08-07 08:53:39,311 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:53:39,311 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398479311 2023-08-07 08:53:39,311 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398539311 2023-08-07 08:53:39,311 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-08-07 08:53:39,311 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=fd83112e7ec055aba3e5a554dab2841e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:39,311 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691398419311"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398419311"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398419311"}]},"ts":"1691398419311"} 2023-08-07 08:53:39,313 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure fd83112e7ec055aba3e5a554dab2841e, server=jenkins-hbase4.apache.org,34559,1691398396467}] 2023-08-07 08:53:39,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. 2023-08-07 08:53:39,468 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => fd83112e7ec055aba3e5a554dab2841e, NAME => 'testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.', STARTKEY => '', ENDKEY => ''} 2023-08-07 08:53:39,469 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-07 08:53:39,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:39,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:53:39,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:39,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:39,470 INFO [StoreOpener-fd83112e7ec055aba3e5a554dab2841e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:39,471 DEBUG [StoreOpener-fd83112e7ec055aba3e5a554dab2841e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1 2023-08-07 08:53:39,471 DEBUG [StoreOpener-fd83112e7ec055aba3e5a554dab2841e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1 2023-08-07 08:53:39,471 INFO [StoreOpener-fd83112e7ec055aba3e5a554dab2841e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fd83112e7ec055aba3e5a554dab2841e columnFamilyName cf1 2023-08-07 08:53:39,478 DEBUG [StoreOpener-fd83112e7ec055aba3e5a554dab2841e-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/143645e1480743c09a8dbb82da50c40b 2023-08-07 08:53:39,481 DEBUG [StoreOpener-fd83112e7ec055aba3e5a554dab2841e-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/2489c9607b2645d5845bd72a39dbb77b 2023-08-07 08:53:39,484 DEBUG [StoreOpener-fd83112e7ec055aba3e5a554dab2841e-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/42ed52d5acc545a89cbdfc54492d824e 2023-08-07 08:53:39,487 DEBUG [StoreOpener-fd83112e7ec055aba3e5a554dab2841e-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/6b8a137fb9644c108395222874065f46 2023-08-07 08:53:39,490 DEBUG [StoreOpener-fd83112e7ec055aba3e5a554dab2841e-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/77fb8078d2014f87887d92cddcb5d75e 2023-08-07 08:53:39,493 DEBUG [StoreOpener-fd83112e7ec055aba3e5a554dab2841e-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/a2e4aab8fc0f4f01bed8c08d5e21e5ad 2023-08-07 08:53:39,496 DEBUG [StoreOpener-fd83112e7ec055aba3e5a554dab2841e-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/ac88c845249d46e49f0532262366d399 2023-08-07 08:53:39,496 INFO [StoreOpener-fd83112e7ec055aba3e5a554dab2841e-1] regionserver.HStore(310): Store=fd83112e7ec055aba3e5a554dab2841e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:39,496 INFO [StoreOpener-fd83112e7ec055aba3e5a554dab2841e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:39,497 DEBUG [StoreOpener-fd83112e7ec055aba3e5a554dab2841e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2 2023-08-07 08:53:39,497 DEBUG [StoreOpener-fd83112e7ec055aba3e5a554dab2841e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2 2023-08-07 08:53:39,498 INFO [StoreOpener-fd83112e7ec055aba3e5a554dab2841e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fd83112e7ec055aba3e5a554dab2841e columnFamilyName cf2 2023-08-07 08:53:39,505 DEBUG [StoreOpener-fd83112e7ec055aba3e5a554dab2841e-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/1ca97b2be41f490591a5050a91d41fa9 2023-08-07 08:53:39,508 DEBUG [StoreOpener-fd83112e7ec055aba3e5a554dab2841e-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/acf834d7b8cb41aea32f6e17233515cf 2023-08-07 08:53:39,512 DEBUG [StoreOpener-fd83112e7ec055aba3e5a554dab2841e-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/da3a5ec77c6c42c4a4be4511e65db6bc 2023-08-07 08:53:39,516 DEBUG [StoreOpener-fd83112e7ec055aba3e5a554dab2841e-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/de611d4e2cf544928b1f9d901618602a 2023-08-07 08:53:39,519 DEBUG [StoreOpener-fd83112e7ec055aba3e5a554dab2841e-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/e328868cd54b41f393a38958421a6b19 2023-08-07 08:53:39,522 DEBUG [StoreOpener-fd83112e7ec055aba3e5a554dab2841e-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/ec5dd4ae9931452288493d04a2a1502d 2023-08-07 08:53:39,526 DEBUG [StoreOpener-fd83112e7ec055aba3e5a554dab2841e-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/f5df38fe79b345d783c503b0e7ebbd90 2023-08-07 08:53:39,526 INFO [StoreOpener-fd83112e7ec055aba3e5a554dab2841e-1] regionserver.HStore(310): Store=fd83112e7ec055aba3e5a554dab2841e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:39,527 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:39,528 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:39,530 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/.splits 2023-08-07 08:53:39,531 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-07 08:53:39,532 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:39,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened fd83112e7ec055aba3e5a554dab2841e; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60065996, jitterRate=-0.10494691133499146}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-07 08:53:39,533 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for fd83112e7ec055aba3e5a554dab2841e: 2023-08-07 08:53:39,534 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e., pid=48, masterSystemTime=1691398419465 2023-08-07 08:53:39,534 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:53:39,534 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-07 08:53:39,535 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-07 08:53:39,535 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-07 08:53:39,536 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-08-07 08:53:39,536 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-08-07 08:53:39,536 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HStore(1912): fd83112e7ec055aba3e5a554dab2841e/cf2 is initiating minor compaction (all files) 2023-08-07 08:53:39,536 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HStore(1912): fd83112e7ec055aba3e5a554dab2841e/cf1 is initiating minor compaction (all files) 2023-08-07 08:53:39,536 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HRegion(2259): Starting compaction of fd83112e7ec055aba3e5a554dab2841e/cf2 in testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. 2023-08-07 08:53:39,536 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of fd83112e7ec055aba3e5a554dab2841e/cf1 in testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. 2023-08-07 08:53:39,537 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/f5df38fe79b345d783c503b0e7ebbd90, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/acf834d7b8cb41aea32f6e17233515cf, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/da3a5ec77c6c42c4a4be4511e65db6bc, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/1ca97b2be41f490591a5050a91d41fa9, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/de611d4e2cf544928b1f9d901618602a, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/ec5dd4ae9931452288493d04a2a1502d, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/e328868cd54b41f393a38958421a6b19] into tmpdir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/.tmp, totalSize=34.9 K 2023-08-07 08:53:39,537 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/a2e4aab8fc0f4f01bed8c08d5e21e5ad, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/2489c9607b2645d5845bd72a39dbb77b, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/6b8a137fb9644c108395222874065f46, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/ac88c845249d46e49f0532262366d399, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/77fb8078d2014f87887d92cddcb5d75e, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/143645e1480743c09a8dbb82da50c40b, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/42ed52d5acc545a89cbdfc54492d824e] into tmpdir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/.tmp, totalSize=34.9 K 2023-08-07 08:53:39,537 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. 2023-08-07 08:53:39,537 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. 2023-08-07 08:53:39,537 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting f5df38fe79b345d783c503b0e7ebbd90, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1691398411745 2023-08-07 08:53:39,537 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting a2e4aab8fc0f4f01bed8c08d5e21e5ad, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1691398411745 2023-08-07 08:53:39,538 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting acf834d7b8cb41aea32f6e17233515cf, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1691398411943 2023-08-07 08:53:39,538 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting 2489c9607b2645d5845bd72a39dbb77b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1691398411943 2023-08-07 08:53:39,538 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=fd83112e7ec055aba3e5a554dab2841e, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:39,538 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting da3a5ec77c6c42c4a4be4511e65db6bc, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1691398412145 2023-08-07 08:53:39,538 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting 6b8a137fb9644c108395222874065f46, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1691398412145 2023-08-07 08:53:39,539 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691398419538"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691398419538"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691398419538"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398419538"}]},"ts":"1691398419538"} 2023-08-07 08:53:39,539 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting 1ca97b2be41f490591a5050a91d41fa9, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1691398412358 2023-08-07 08:53:39,539 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting ac88c845249d46e49f0532262366d399, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1691398412358 2023-08-07 08:53:39,539 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting de611d4e2cf544928b1f9d901618602a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1691398412545 2023-08-07 08:53:39,539 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting 77fb8078d2014f87887d92cddcb5d75e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1691398412545 2023-08-07 08:53:39,540 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting ec5dd4ae9931452288493d04a2a1502d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1691398412734 2023-08-07 08:53:39,540 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting 143645e1480743c09a8dbb82da50c40b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1691398412734 2023-08-07 08:53:39,540 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting e328868cd54b41f393a38958421a6b19, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1691398413270 2023-08-07 08:53:39,540 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting 42ed52d5acc545a89cbdfc54492d824e, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1691398413270 2023-08-07 08:53:39,544 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=48, resume processing ppid=47 2023-08-07 08:53:39,544 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure fd83112e7ec055aba3e5a554dab2841e, server=jenkins-hbase4.apache.org,34559,1691398396467 in 227 msec 2023-08-07 08:53:39,545 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN in 1.5640 sec 2023-08-07 08:53:39,559 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] throttle.PressureAwareThroughputController(145): fd83112e7ec055aba3e5a554dab2841e#cf1#compaction#50 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-07 08:53:39,567 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] throttle.PressureAwareThroughputController(145): fd83112e7ec055aba3e5a554dab2841e#cf2#compaction#51 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-07 08:53:39,579 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/.tmp/cf1/3042d7ae54f044d89583e88524b49e66 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/3042d7ae54f044d89583e88524b49e66 2023-08-07 08:53:39,584 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in fd83112e7ec055aba3e5a554dab2841e/cf1 of fd83112e7ec055aba3e5a554dab2841e into 3042d7ae54f044d89583e88524b49e66(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 08:53:39,584 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for fd83112e7ec055aba3e5a554dab2841e: 2023-08-07 08:53:39,584 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e., storeName=fd83112e7ec055aba3e5a554dab2841e/cf1, priority=9, startTime=1691398419534; duration=0sec 2023-08-07 08:53:39,586 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:53:39,594 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/.tmp/cf2/f28c92893ac942e6a983f8ea2aa0c418 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/f28c92893ac942e6a983f8ea2aa0c418 2023-08-07 08:53:39,599 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in fd83112e7ec055aba3e5a554dab2841e/cf2 of fd83112e7ec055aba3e5a554dab2841e into f28c92893ac942e6a983f8ea2aa0c418(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 08:53:39,599 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for fd83112e7ec055aba3e5a554dab2841e: 2023-08-07 08:53:39,599 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e., storeName=fd83112e7ec055aba3e5a554dab2841e/cf2, priority=9, startTime=1691398419534; duration=0sec 2023-08-07 08:53:39,600 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:53:39,888 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-08-07 08:53:39,890 DEBUG [Listener at localhost/33561] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-08-07 08:53:39,892 INFO [Listener at localhost/33561] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-08-07 08:53:39,894 INFO [Listener at localhost/33561] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-08-07 08:53:39,895 INFO [Listener at localhost/33561] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-08-07 08:53:39,895 INFO [Listener at localhost/33561] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-08-07 08:53:39,908 WARN [Listener at localhost/33561] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-07 08:53:39,910 INFO [Listener at localhost/33561] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-08-07 08:53:39,910 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testRollbackAndDoubleExecution 2023-08-07 08:53:39,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:53:39,914 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=49 2023-08-07 08:53:39,914 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398419914"}]},"ts":"1691398419914"} 2023-08-07 08:53:39,915 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-08-07 08:53:39,917 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-08-07 08:53:39,918 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN}] 2023-08-07 08:53:39,918 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN 2023-08-07 08:53:39,919 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=fd83112e7ec055aba3e5a554dab2841e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:39,919 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691398419919"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398419919"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398419919"}]},"ts":"1691398419919"} 2023-08-07 08:53:39,920 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure fd83112e7ec055aba3e5a554dab2841e, server=jenkins-hbase4.apache.org,34559,1691398396467}] 2023-08-07 08:53:40,015 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=49 2023-08-07 08:53:40,072 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:40,072 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing fd83112e7ec055aba3e5a554dab2841e, disabling compactions & flushes 2023-08-07 08:53:40,072 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. 2023-08-07 08:53:40,072 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. 2023-08-07 08:53:40,072 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. after waiting 0 ms 2023-08-07 08:53:40,072 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. 2023-08-07 08:53:40,073 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/a2e4aab8fc0f4f01bed8c08d5e21e5ad, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/2489c9607b2645d5845bd72a39dbb77b, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/6b8a137fb9644c108395222874065f46, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/ac88c845249d46e49f0532262366d399, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/77fb8078d2014f87887d92cddcb5d75e, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/143645e1480743c09a8dbb82da50c40b, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/42ed52d5acc545a89cbdfc54492d824e] to archive 2023-08-07 08:53:40,074 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-07 08:53:40,076 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/a2e4aab8fc0f4f01bed8c08d5e21e5ad to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/a2e4aab8fc0f4f01bed8c08d5e21e5ad 2023-08-07 08:53:40,077 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/2489c9607b2645d5845bd72a39dbb77b to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/2489c9607b2645d5845bd72a39dbb77b 2023-08-07 08:53:40,078 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/6b8a137fb9644c108395222874065f46 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/6b8a137fb9644c108395222874065f46 2023-08-07 08:53:40,079 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/ac88c845249d46e49f0532262366d399 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/ac88c845249d46e49f0532262366d399 2023-08-07 08:53:40,081 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/77fb8078d2014f87887d92cddcb5d75e to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/77fb8078d2014f87887d92cddcb5d75e 2023-08-07 08:53:40,082 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/143645e1480743c09a8dbb82da50c40b to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/143645e1480743c09a8dbb82da50c40b 2023-08-07 08:53:40,084 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/42ed52d5acc545a89cbdfc54492d824e to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/42ed52d5acc545a89cbdfc54492d824e 2023-08-07 08:53:40,090 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/f5df38fe79b345d783c503b0e7ebbd90, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/acf834d7b8cb41aea32f6e17233515cf, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/da3a5ec77c6c42c4a4be4511e65db6bc, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/1ca97b2be41f490591a5050a91d41fa9, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/de611d4e2cf544928b1f9d901618602a, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/ec5dd4ae9931452288493d04a2a1502d, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/e328868cd54b41f393a38958421a6b19] to archive 2023-08-07 08:53:40,091 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-07 08:53:40,093 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/f5df38fe79b345d783c503b0e7ebbd90 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/f5df38fe79b345d783c503b0e7ebbd90 2023-08-07 08:53:40,094 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/acf834d7b8cb41aea32f6e17233515cf to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/acf834d7b8cb41aea32f6e17233515cf 2023-08-07 08:53:40,096 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/da3a5ec77c6c42c4a4be4511e65db6bc to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/da3a5ec77c6c42c4a4be4511e65db6bc 2023-08-07 08:53:40,097 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/1ca97b2be41f490591a5050a91d41fa9 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/1ca97b2be41f490591a5050a91d41fa9 2023-08-07 08:53:40,098 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/de611d4e2cf544928b1f9d901618602a to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/de611d4e2cf544928b1f9d901618602a 2023-08-07 08:53:40,100 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/ec5dd4ae9931452288493d04a2a1502d to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/ec5dd4ae9931452288493d04a2a1502d 2023-08-07 08:53:40,101 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/e328868cd54b41f393a38958421a6b19 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/e328868cd54b41f393a38958421a6b19 2023-08-07 08:53:40,105 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-08-07 08:53:40,105 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-07 08:53:40,105 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e. 2023-08-07 08:53:40,105 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for fd83112e7ec055aba3e5a554dab2841e: 2023-08-07 08:53:40,107 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:40,107 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=fd83112e7ec055aba3e5a554dab2841e, regionState=CLOSED 2023-08-07 08:53:40,107 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691398420107"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398420107"}]},"ts":"1691398420107"} 2023-08-07 08:53:40,111 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-08-07 08:53:40,111 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure fd83112e7ec055aba3e5a554dab2841e, server=jenkins-hbase4.apache.org,34559,1691398396467 in 189 msec 2023-08-07 08:53:40,112 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-08-07 08:53:40,112 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, UNASSIGN in 194 msec 2023-08-07 08:53:40,113 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398420113"}]},"ts":"1691398420113"} 2023-08-07 08:53:40,114 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-08-07 08:53:40,116 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-08-07 08:53:40,117 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 206 msec 2023-08-07 08:53:40,216 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=49 2023-08-07 08:53:40,216 INFO [Listener at localhost/33561] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-08-07 08:53:40,217 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testRollbackAndDoubleExecution 2023-08-07 08:53:40,218 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:53:40,220 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:53:40,220 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:53:40,221 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=52 2023-08-07 08:53:40,224 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:40,226 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/recovered.edits] 2023-08-07 08:53:40,228 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/3042d7ae54f044d89583e88524b49e66 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf1/3042d7ae54f044d89583e88524b49e66 2023-08-07 08:53:40,231 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/f28c92893ac942e6a983f8ea2aa0c418 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/cf2/f28c92893ac942e6a983f8ea2aa0c418 2023-08-07 08:53:40,233 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/recovered.edits/90.seqid to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e/recovered.edits/90.seqid 2023-08-07 08:53:40,234 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRollbackAndDoubleExecution/fd83112e7ec055aba3e5a554dab2841e 2023-08-07 08:53:40,234 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-08-07 08:53:40,237 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:53:40,239 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-08-07 08:53:40,241 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-08-07 08:53:40,242 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:53:40,242 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-08-07 08:53:40,242 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691398420242"}]},"ts":"9223372036854775807"} 2023-08-07 08:53:40,244 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-08-07 08:53:40,244 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => fd83112e7ec055aba3e5a554dab2841e, NAME => 'testRollbackAndDoubleExecution,,1691398411240.fd83112e7ec055aba3e5a554dab2841e.', STARTKEY => '', ENDKEY => ''}] 2023-08-07 08:53:40,244 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-08-07 08:53:40,244 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1691398420244"}]},"ts":"9223372036854775807"} 2023-08-07 08:53:40,246 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-08-07 08:53:40,248 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:53:40,249 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 31 msec 2023-08-07 08:53:40,322 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=52 2023-08-07 08:53:40,322 INFO [Listener at localhost/33561] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-08-07 08:53:40,339 INFO [Listener at localhost/33561] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=398 (was 397) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1896775705_17 at /127.0.0.1:56852 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1896775705_17 at /127.0.0.1:33698 [Waiting for operation #38] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1896775705_17 at /127.0.0.1:53288 [Waiting for operation #38] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1896775705_17 at /127.0.0.1:57448 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1896775705_17 at /127.0.0.1:41420 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x36ae5be-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:2;jenkins-hbase4:34559-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1896775705_17 at /127.0.0.1:57054 [Waiting for operation #28] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1896775705_17 at /127.0.0.1:41812 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1896775705_17 at /127.0.0.1:57436 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1896775705_17 at /127.0.0.1:53904 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/cluster_383c75a2-fb83-c721-ca60-8c2ce8eb403c/dfs/data/data6/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/cluster_383c75a2-fb83-c721-ca60-8c2ce8eb403c/dfs/data/data5/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1896775705_17 at /127.0.0.1:50860 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1896775705_17 at /127.0.0.1:56870 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1896775705_17 at /127.0.0.1:41412 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1896775705_17 at /127.0.0.1:57554 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1896775705_17 at /127.0.0.1:50878 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1896775705_17 at /127.0.0.1:53994 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1896775705_17 at /127.0.0.1:41248 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x36ae5be-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=857 (was 873), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=368 (was 408), ProcessCount=174 (was 174), AvailableMemoryMB=1923 (was 2015) 2023-08-07 08:53:40,356 INFO [Listener at localhost/33561] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=398, OpenFileDescriptor=857, MaxFileDescriptor=60000, SystemLoadAverage=368, ProcessCount=174, AvailableMemoryMB=1922 2023-08-07 08:53:40,356 WARN [Listener at localhost/33561] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-07 08:53:40,359 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-08-07 08:53:40,359 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-08-07 08:53:40,361 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:53:40,363 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-08-07 08:53:40,363 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-07 08:53:40,365 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionNoStoreFile/fd6d700fb2bf2b02eb0b469b2b0d401b 2023-08-07 08:53:40,365 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionNoStoreFile/fd6d700fb2bf2b02eb0b469b2b0d401b empty. 2023-08-07 08:53:40,366 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionNoStoreFile/fd6d700fb2bf2b02eb0b469b2b0d401b 2023-08-07 08:53:40,366 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-08-07 08:53:40,376 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-08-07 08:53:40,377 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7675): creating {ENCODED => fd6d700fb2bf2b02eb0b469b2b0d401b, NAME => 'testSplitTableRegionNoStoreFile,,1691398420360.fd6d700fb2bf2b02eb0b469b2b0d401b.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp 2023-08-07 08:53:40,384 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1691398420360.fd6d700fb2bf2b02eb0b469b2b0d401b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:53:40,384 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1604): Closing fd6d700fb2bf2b02eb0b469b2b0d401b, disabling compactions & flushes 2023-08-07 08:53:40,384 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1691398420360.fd6d700fb2bf2b02eb0b469b2b0d401b. 2023-08-07 08:53:40,384 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1691398420360.fd6d700fb2bf2b02eb0b469b2b0d401b. 2023-08-07 08:53:40,384 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1691398420360.fd6d700fb2bf2b02eb0b469b2b0d401b. after waiting 0 ms 2023-08-07 08:53:40,384 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1691398420360.fd6d700fb2bf2b02eb0b469b2b0d401b. 2023-08-07 08:53:40,384 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1691398420360.fd6d700fb2bf2b02eb0b469b2b0d401b. 2023-08-07 08:53:40,384 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1558): Region close journal for fd6d700fb2bf2b02eb0b469b2b0d401b: 2023-08-07 08:53:40,386 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-08-07 08:53:40,387 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1691398420360.fd6d700fb2bf2b02eb0b469b2b0d401b.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1691398420387"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398420387"}]},"ts":"1691398420387"} 2023-08-07 08:53:40,389 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-07 08:53:40,390 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-07 08:53:40,390 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398420390"}]},"ts":"1691398420390"} 2023-08-07 08:53:40,391 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-08-07 08:53:40,394 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-07 08:53:40,395 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-07 08:53:40,395 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-07 08:53:40,395 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-07 08:53:40,395 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-07 08:53:40,395 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=fd6d700fb2bf2b02eb0b469b2b0d401b, ASSIGN}] 2023-08-07 08:53:40,396 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=fd6d700fb2bf2b02eb0b469b2b0d401b, ASSIGN 2023-08-07 08:53:40,397 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=fd6d700fb2bf2b02eb0b469b2b0d401b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34559,1691398396467; forceNewPlan=false, retain=false 2023-08-07 08:53:40,547 INFO [jenkins-hbase4:33357] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-07 08:53:40,548 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=fd6d700fb2bf2b02eb0b469b2b0d401b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:40,548 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1691398420360.fd6d700fb2bf2b02eb0b469b2b0d401b.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1691398420548"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398420548"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398420548"}]},"ts":"1691398420548"} 2023-08-07 08:53:40,550 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure fd6d700fb2bf2b02eb0b469b2b0d401b, server=jenkins-hbase4.apache.org,34559,1691398396467}] 2023-08-07 08:53:40,705 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionNoStoreFile,,1691398420360.fd6d700fb2bf2b02eb0b469b2b0d401b. 2023-08-07 08:53:40,705 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => fd6d700fb2bf2b02eb0b469b2b0d401b, NAME => 'testSplitTableRegionNoStoreFile,,1691398420360.fd6d700fb2bf2b02eb0b469b2b0d401b.', STARTKEY => '', ENDKEY => ''} 2023-08-07 08:53:40,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-07 08:53:40,706 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile fd6d700fb2bf2b02eb0b469b2b0d401b 2023-08-07 08:53:40,706 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1691398420360.fd6d700fb2bf2b02eb0b469b2b0d401b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:53:40,706 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for fd6d700fb2bf2b02eb0b469b2b0d401b 2023-08-07 08:53:40,706 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for fd6d700fb2bf2b02eb0b469b2b0d401b 2023-08-07 08:53:40,707 INFO [StoreOpener-fd6d700fb2bf2b02eb0b469b2b0d401b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region fd6d700fb2bf2b02eb0b469b2b0d401b 2023-08-07 08:53:40,714 DEBUG [StoreOpener-fd6d700fb2bf2b02eb0b469b2b0d401b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionNoStoreFile/fd6d700fb2bf2b02eb0b469b2b0d401b/cf1 2023-08-07 08:53:40,714 DEBUG [StoreOpener-fd6d700fb2bf2b02eb0b469b2b0d401b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionNoStoreFile/fd6d700fb2bf2b02eb0b469b2b0d401b/cf1 2023-08-07 08:53:40,715 INFO [StoreOpener-fd6d700fb2bf2b02eb0b469b2b0d401b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fd6d700fb2bf2b02eb0b469b2b0d401b columnFamilyName cf1 2023-08-07 08:53:40,715 INFO [StoreOpener-fd6d700fb2bf2b02eb0b469b2b0d401b-1] regionserver.HStore(310): Store=fd6d700fb2bf2b02eb0b469b2b0d401b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:40,716 INFO [StoreOpener-fd6d700fb2bf2b02eb0b469b2b0d401b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region fd6d700fb2bf2b02eb0b469b2b0d401b 2023-08-07 08:53:40,717 DEBUG [StoreOpener-fd6d700fb2bf2b02eb0b469b2b0d401b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionNoStoreFile/fd6d700fb2bf2b02eb0b469b2b0d401b/cf2 2023-08-07 08:53:40,717 DEBUG [StoreOpener-fd6d700fb2bf2b02eb0b469b2b0d401b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionNoStoreFile/fd6d700fb2bf2b02eb0b469b2b0d401b/cf2 2023-08-07 08:53:40,718 INFO [StoreOpener-fd6d700fb2bf2b02eb0b469b2b0d401b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fd6d700fb2bf2b02eb0b469b2b0d401b columnFamilyName cf2 2023-08-07 08:53:40,719 INFO [StoreOpener-fd6d700fb2bf2b02eb0b469b2b0d401b-1] regionserver.HStore(310): Store=fd6d700fb2bf2b02eb0b469b2b0d401b/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:40,719 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionNoStoreFile/fd6d700fb2bf2b02eb0b469b2b0d401b 2023-08-07 08:53:40,720 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionNoStoreFile/fd6d700fb2bf2b02eb0b469b2b0d401b 2023-08-07 08:53:40,722 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-07 08:53:40,723 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for fd6d700fb2bf2b02eb0b469b2b0d401b 2023-08-07 08:53:40,725 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionNoStoreFile/fd6d700fb2bf2b02eb0b469b2b0d401b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 08:53:40,726 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened fd6d700fb2bf2b02eb0b469b2b0d401b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60914684, jitterRate=-0.09230047464370728}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-07 08:53:40,726 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for fd6d700fb2bf2b02eb0b469b2b0d401b: 2023-08-07 08:53:40,726 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1691398420360.fd6d700fb2bf2b02eb0b469b2b0d401b., pid=55, masterSystemTime=1691398420702 2023-08-07 08:53:40,728 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1691398420360.fd6d700fb2bf2b02eb0b469b2b0d401b. 2023-08-07 08:53:40,728 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionNoStoreFile,,1691398420360.fd6d700fb2bf2b02eb0b469b2b0d401b. 2023-08-07 08:53:40,729 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=fd6d700fb2bf2b02eb0b469b2b0d401b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:40,729 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1691398420360.fd6d700fb2bf2b02eb0b469b2b0d401b.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1691398420729"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691398420729"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691398420729"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398420729"}]},"ts":"1691398420729"} 2023-08-07 08:53:40,732 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=54 2023-08-07 08:53:40,732 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure fd6d700fb2bf2b02eb0b469b2b0d401b, server=jenkins-hbase4.apache.org,34559,1691398396467 in 180 msec 2023-08-07 08:53:40,734 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=53 2023-08-07 08:53:40,734 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=fd6d700fb2bf2b02eb0b469b2b0d401b, ASSIGN in 337 msec 2023-08-07 08:53:40,735 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-07 08:53:40,735 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398420735"}]},"ts":"1691398420735"} 2023-08-07 08:53:40,736 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-08-07 08:53:40,738 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-08-07 08:53:40,740 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 379 msec 2023-08-07 08:53:40,862 DEBUG [Listener at localhost/33561] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:40,863 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(1029): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=fd6d700fb2bf2b02eb0b469b2b0d401b, daughterA=9ca78a73cef89f541dcf91876dbfafd9, daughterB=5b3565e5e309ccbd325fa8dc34e561c4 2023-08-07 08:53:40,864 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=fd6d700fb2bf2b02eb0b469b2b0d401b, daughterA=9ca78a73cef89f541dcf91876dbfafd9, daughterB=5b3565e5e309ccbd325fa8dc34e561c4 2023-08-07 08:53:40,864 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=fd6d700fb2bf2b02eb0b469b2b0d401b, daughterA=9ca78a73cef89f541dcf91876dbfafd9, daughterB=5b3565e5e309ccbd325fa8dc34e561c4 2023-08-07 08:53:40,864 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=fd6d700fb2bf2b02eb0b469b2b0d401b, daughterA=9ca78a73cef89f541dcf91876dbfafd9, daughterB=5b3565e5e309ccbd325fa8dc34e561c4 2023-08-07 08:53:40,866 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=fd6d700fb2bf2b02eb0b469b2b0d401b, UNASSIGN}] 2023-08-07 08:53:40,867 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=fd6d700fb2bf2b02eb0b469b2b0d401b, UNASSIGN 2023-08-07 08:53:40,868 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=fd6d700fb2bf2b02eb0b469b2b0d401b, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:40,868 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1691398420360.fd6d700fb2bf2b02eb0b469b2b0d401b.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1691398420868"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398420868"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398420868"}]},"ts":"1691398420868"} 2023-08-07 08:53:40,870 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure fd6d700fb2bf2b02eb0b469b2b0d401b, server=jenkins-hbase4.apache.org,34559,1691398396467}] 2023-08-07 08:53:41,022 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close fd6d700fb2bf2b02eb0b469b2b0d401b 2023-08-07 08:53:41,023 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing fd6d700fb2bf2b02eb0b469b2b0d401b, disabling compactions & flushes 2023-08-07 08:53:41,023 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1691398420360.fd6d700fb2bf2b02eb0b469b2b0d401b. 2023-08-07 08:53:41,023 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1691398420360.fd6d700fb2bf2b02eb0b469b2b0d401b. 2023-08-07 08:53:41,023 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1691398420360.fd6d700fb2bf2b02eb0b469b2b0d401b. after waiting 0 ms 2023-08-07 08:53:41,023 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1691398420360.fd6d700fb2bf2b02eb0b469b2b0d401b. 2023-08-07 08:53:41,027 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionNoStoreFile/fd6d700fb2bf2b02eb0b469b2b0d401b/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-08-07 08:53:41,028 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-07 08:53:41,028 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1691398420360.fd6d700fb2bf2b02eb0b469b2b0d401b. 2023-08-07 08:53:41,028 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for fd6d700fb2bf2b02eb0b469b2b0d401b: 2023-08-07 08:53:41,037 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed fd6d700fb2bf2b02eb0b469b2b0d401b 2023-08-07 08:53:41,037 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=fd6d700fb2bf2b02eb0b469b2b0d401b, regionState=CLOSED 2023-08-07 08:53:41,038 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1691398420360.fd6d700fb2bf2b02eb0b469b2b0d401b.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1691398421037"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398421037"}]},"ts":"1691398421037"} 2023-08-07 08:53:41,041 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=57 2023-08-07 08:53:41,041 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure fd6d700fb2bf2b02eb0b469b2b0d401b, server=jenkins-hbase4.apache.org,34559,1691398396467 in 169 msec 2023-08-07 08:53:41,043 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-08-07 08:53:41,043 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=fd6d700fb2bf2b02eb0b469b2b0d401b, UNASSIGN in 175 msec 2023-08-07 08:53:41,083 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionNoStoreFile/9ca78a73cef89f541dcf91876dbfafd9/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-08-07 08:53:41,085 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionNoStoreFile/5b3565e5e309ccbd325fa8dc34e561c4/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-08-07 08:53:41,087 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1691398420360.fd6d700fb2bf2b02eb0b469b2b0d401b.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1691398421087"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1691398421087"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1691398421087"}]},"ts":"1691398421087"} 2023-08-07 08:53:41,087 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1691398420862.9ca78a73cef89f541dcf91876dbfafd9.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1691398421087"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398421087"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398421087"}]},"ts":"1691398421087"} 2023-08-07 08:53:41,087 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1691398420862.5b3565e5e309ccbd325fa8dc34e561c4.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1691398421087"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398421087"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398421087"}]},"ts":"1691398421087"} 2023-08-07 08:53:41,091 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=9ca78a73cef89f541dcf91876dbfafd9, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=5b3565e5e309ccbd325fa8dc34e561c4, ASSIGN}] 2023-08-07 08:53:41,092 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=5b3565e5e309ccbd325fa8dc34e561c4, ASSIGN 2023-08-07 08:53:41,093 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=5b3565e5e309ccbd325fa8dc34e561c4, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,34559,1691398396467; forceNewPlan=false, retain=false 2023-08-07 08:53:41,094 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=9ca78a73cef89f541dcf91876dbfafd9, ASSIGN 2023-08-07 08:53:41,094 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=9ca78a73cef89f541dcf91876dbfafd9, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,34559,1691398396467; forceNewPlan=false, retain=false 2023-08-07 08:53:41,243 INFO [jenkins-hbase4:33357] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-08-07 08:53:41,245 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=9ca78a73cef89f541dcf91876dbfafd9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:41,245 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1691398420862.9ca78a73cef89f541dcf91876dbfafd9.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1691398421245"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398421245"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398421245"}]},"ts":"1691398421245"} 2023-08-07 08:53:41,247 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=59, state=RUNNABLE; OpenRegionProcedure 9ca78a73cef89f541dcf91876dbfafd9, server=jenkins-hbase4.apache.org,34559,1691398396467}] 2023-08-07 08:53:41,249 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=5b3565e5e309ccbd325fa8dc34e561c4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:41,249 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1691398420862.5b3565e5e309ccbd325fa8dc34e561c4.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1691398421249"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398421249"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398421249"}]},"ts":"1691398421249"} 2023-08-07 08:53:41,250 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=60, state=RUNNABLE; OpenRegionProcedure 5b3565e5e309ccbd325fa8dc34e561c4, server=jenkins-hbase4.apache.org,34559,1691398396467}] 2023-08-07 08:53:41,401 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionNoStoreFile,41,1691398420862.5b3565e5e309ccbd325fa8dc34e561c4. 2023-08-07 08:53:41,402 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 5b3565e5e309ccbd325fa8dc34e561c4, NAME => 'testSplitTableRegionNoStoreFile,41,1691398420862.5b3565e5e309ccbd325fa8dc34e561c4.', STARTKEY => '41', ENDKEY => ''} 2023-08-07 08:53:41,402 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-07 08:53:41,402 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 5b3565e5e309ccbd325fa8dc34e561c4 2023-08-07 08:53:41,402 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,41,1691398420862.5b3565e5e309ccbd325fa8dc34e561c4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:53:41,402 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 5b3565e5e309ccbd325fa8dc34e561c4 2023-08-07 08:53:41,402 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 5b3565e5e309ccbd325fa8dc34e561c4 2023-08-07 08:53:41,403 INFO [StoreOpener-5b3565e5e309ccbd325fa8dc34e561c4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 5b3565e5e309ccbd325fa8dc34e561c4 2023-08-07 08:53:41,405 DEBUG [StoreOpener-5b3565e5e309ccbd325fa8dc34e561c4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionNoStoreFile/5b3565e5e309ccbd325fa8dc34e561c4/cf1 2023-08-07 08:53:41,405 DEBUG [StoreOpener-5b3565e5e309ccbd325fa8dc34e561c4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionNoStoreFile/5b3565e5e309ccbd325fa8dc34e561c4/cf1 2023-08-07 08:53:41,405 INFO [StoreOpener-5b3565e5e309ccbd325fa8dc34e561c4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5b3565e5e309ccbd325fa8dc34e561c4 columnFamilyName cf1 2023-08-07 08:53:41,406 INFO [StoreOpener-5b3565e5e309ccbd325fa8dc34e561c4-1] regionserver.HStore(310): Store=5b3565e5e309ccbd325fa8dc34e561c4/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:41,406 INFO [StoreOpener-5b3565e5e309ccbd325fa8dc34e561c4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 5b3565e5e309ccbd325fa8dc34e561c4 2023-08-07 08:53:41,407 DEBUG [StoreOpener-5b3565e5e309ccbd325fa8dc34e561c4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionNoStoreFile/5b3565e5e309ccbd325fa8dc34e561c4/cf2 2023-08-07 08:53:41,407 DEBUG [StoreOpener-5b3565e5e309ccbd325fa8dc34e561c4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionNoStoreFile/5b3565e5e309ccbd325fa8dc34e561c4/cf2 2023-08-07 08:53:41,407 INFO [StoreOpener-5b3565e5e309ccbd325fa8dc34e561c4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5b3565e5e309ccbd325fa8dc34e561c4 columnFamilyName cf2 2023-08-07 08:53:41,408 INFO [StoreOpener-5b3565e5e309ccbd325fa8dc34e561c4-1] regionserver.HStore(310): Store=5b3565e5e309ccbd325fa8dc34e561c4/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:41,409 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionNoStoreFile/5b3565e5e309ccbd325fa8dc34e561c4 2023-08-07 08:53:41,410 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionNoStoreFile/5b3565e5e309ccbd325fa8dc34e561c4 2023-08-07 08:53:41,412 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-07 08:53:41,413 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 5b3565e5e309ccbd325fa8dc34e561c4 2023-08-07 08:53:41,414 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 5b3565e5e309ccbd325fa8dc34e561c4; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71737793, jitterRate=0.06897641718387604}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-07 08:53:41,414 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 5b3565e5e309ccbd325fa8dc34e561c4: 2023-08-07 08:53:41,414 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1691398420862.5b3565e5e309ccbd325fa8dc34e561c4., pid=62, masterSystemTime=1691398421398 2023-08-07 08:53:41,416 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1691398420862.5b3565e5e309ccbd325fa8dc34e561c4. 2023-08-07 08:53:41,416 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionNoStoreFile,41,1691398420862.5b3565e5e309ccbd325fa8dc34e561c4. 2023-08-07 08:53:41,416 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionNoStoreFile,,1691398420862.9ca78a73cef89f541dcf91876dbfafd9. 2023-08-07 08:53:41,416 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 9ca78a73cef89f541dcf91876dbfafd9, NAME => 'testSplitTableRegionNoStoreFile,,1691398420862.9ca78a73cef89f541dcf91876dbfafd9.', STARTKEY => '', ENDKEY => '41'} 2023-08-07 08:53:41,416 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=5b3565e5e309ccbd325fa8dc34e561c4, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:41,416 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-07 08:53:41,416 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1691398420862.5b3565e5e309ccbd325fa8dc34e561c4.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1691398421416"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691398421416"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691398421416"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398421416"}]},"ts":"1691398421416"} 2023-08-07 08:53:41,416 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 9ca78a73cef89f541dcf91876dbfafd9 2023-08-07 08:53:41,416 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1691398420862.9ca78a73cef89f541dcf91876dbfafd9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:53:41,416 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 9ca78a73cef89f541dcf91876dbfafd9 2023-08-07 08:53:41,417 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 9ca78a73cef89f541dcf91876dbfafd9 2023-08-07 08:53:41,418 INFO [StoreOpener-9ca78a73cef89f541dcf91876dbfafd9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 9ca78a73cef89f541dcf91876dbfafd9 2023-08-07 08:53:41,419 DEBUG [StoreOpener-9ca78a73cef89f541dcf91876dbfafd9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionNoStoreFile/9ca78a73cef89f541dcf91876dbfafd9/cf1 2023-08-07 08:53:41,419 DEBUG [StoreOpener-9ca78a73cef89f541dcf91876dbfafd9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionNoStoreFile/9ca78a73cef89f541dcf91876dbfafd9/cf1 2023-08-07 08:53:41,420 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=60 2023-08-07 08:53:41,420 INFO [StoreOpener-9ca78a73cef89f541dcf91876dbfafd9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9ca78a73cef89f541dcf91876dbfafd9 columnFamilyName cf1 2023-08-07 08:53:41,420 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=60, state=SUCCESS; OpenRegionProcedure 5b3565e5e309ccbd325fa8dc34e561c4, server=jenkins-hbase4.apache.org,34559,1691398396467 in 168 msec 2023-08-07 08:53:41,420 INFO [StoreOpener-9ca78a73cef89f541dcf91876dbfafd9-1] regionserver.HStore(310): Store=9ca78a73cef89f541dcf91876dbfafd9/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:41,420 INFO [StoreOpener-9ca78a73cef89f541dcf91876dbfafd9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 9ca78a73cef89f541dcf91876dbfafd9 2023-08-07 08:53:41,421 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=5b3565e5e309ccbd325fa8dc34e561c4, ASSIGN in 329 msec 2023-08-07 08:53:41,422 DEBUG [StoreOpener-9ca78a73cef89f541dcf91876dbfafd9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionNoStoreFile/9ca78a73cef89f541dcf91876dbfafd9/cf2 2023-08-07 08:53:41,422 DEBUG [StoreOpener-9ca78a73cef89f541dcf91876dbfafd9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionNoStoreFile/9ca78a73cef89f541dcf91876dbfafd9/cf2 2023-08-07 08:53:41,422 INFO [StoreOpener-9ca78a73cef89f541dcf91876dbfafd9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9ca78a73cef89f541dcf91876dbfafd9 columnFamilyName cf2 2023-08-07 08:53:41,422 INFO [StoreOpener-9ca78a73cef89f541dcf91876dbfafd9-1] regionserver.HStore(310): Store=9ca78a73cef89f541dcf91876dbfafd9/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:41,423 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionNoStoreFile/9ca78a73cef89f541dcf91876dbfafd9 2023-08-07 08:53:41,424 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionNoStoreFile/9ca78a73cef89f541dcf91876dbfafd9 2023-08-07 08:53:41,425 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-07 08:53:41,426 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 9ca78a73cef89f541dcf91876dbfafd9 2023-08-07 08:53:41,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 9ca78a73cef89f541dcf91876dbfafd9; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64782241, jitterRate=-0.034669384360313416}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-07 08:53:41,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 9ca78a73cef89f541dcf91876dbfafd9: 2023-08-07 08:53:41,428 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1691398420862.9ca78a73cef89f541dcf91876dbfafd9., pid=61, masterSystemTime=1691398421398 2023-08-07 08:53:41,429 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1691398420862.9ca78a73cef89f541dcf91876dbfafd9. 2023-08-07 08:53:41,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionNoStoreFile,,1691398420862.9ca78a73cef89f541dcf91876dbfafd9. 2023-08-07 08:53:41,430 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=9ca78a73cef89f541dcf91876dbfafd9, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:41,430 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1691398420862.9ca78a73cef89f541dcf91876dbfafd9.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1691398421429"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691398421429"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691398421429"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398421429"}]},"ts":"1691398421429"} 2023-08-07 08:53:41,433 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=59 2023-08-07 08:53:41,433 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=59, state=SUCCESS; OpenRegionProcedure 9ca78a73cef89f541dcf91876dbfafd9, server=jenkins-hbase4.apache.org,34559,1691398396467 in 184 msec 2023-08-07 08:53:41,434 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=59, resume processing ppid=56 2023-08-07 08:53:41,434 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=9ca78a73cef89f541dcf91876dbfafd9, ASSIGN in 342 msec 2023-08-07 08:53:41,435 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=fd6d700fb2bf2b02eb0b469b2b0d401b, daughterA=9ca78a73cef89f541dcf91876dbfafd9, daughterB=5b3565e5e309ccbd325fa8dc34e561c4 in 572 msec 2023-08-07 08:53:41,618 DEBUG [Listener at localhost/33561] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-08-07 08:53:41,621 WARN [Listener at localhost/33561] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-07 08:53:41,622 INFO [Listener at localhost/33561] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-08-07 08:53:41,623 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionNoStoreFile 2023-08-07 08:53:41,624 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:53:41,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=63 2023-08-07 08:53:41,626 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398421626"}]},"ts":"1691398421626"} 2023-08-07 08:53:41,628 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-08-07 08:53:41,630 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-08-07 08:53:41,631 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=9ca78a73cef89f541dcf91876dbfafd9, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=5b3565e5e309ccbd325fa8dc34e561c4, UNASSIGN}] 2023-08-07 08:53:41,631 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=5b3565e5e309ccbd325fa8dc34e561c4, UNASSIGN 2023-08-07 08:53:41,632 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=5b3565e5e309ccbd325fa8dc34e561c4, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:41,632 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1691398420862.5b3565e5e309ccbd325fa8dc34e561c4.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1691398421632"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398421632"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398421632"}]},"ts":"1691398421632"} 2023-08-07 08:53:41,633 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure 5b3565e5e309ccbd325fa8dc34e561c4, server=jenkins-hbase4.apache.org,34559,1691398396467}] 2023-08-07 08:53:41,635 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=9ca78a73cef89f541dcf91876dbfafd9, UNASSIGN 2023-08-07 08:53:41,636 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=9ca78a73cef89f541dcf91876dbfafd9, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:41,636 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1691398420862.9ca78a73cef89f541dcf91876dbfafd9.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1691398421636"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398421636"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398421636"}]},"ts":"1691398421636"} 2023-08-07 08:53:41,637 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure 9ca78a73cef89f541dcf91876dbfafd9, server=jenkins-hbase4.apache.org,34559,1691398396467}] 2023-08-07 08:53:41,727 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=63 2023-08-07 08:53:41,785 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 9ca78a73cef89f541dcf91876dbfafd9 2023-08-07 08:53:41,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 9ca78a73cef89f541dcf91876dbfafd9, disabling compactions & flushes 2023-08-07 08:53:41,786 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1691398420862.9ca78a73cef89f541dcf91876dbfafd9. 2023-08-07 08:53:41,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1691398420862.9ca78a73cef89f541dcf91876dbfafd9. 2023-08-07 08:53:41,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1691398420862.9ca78a73cef89f541dcf91876dbfafd9. after waiting 0 ms 2023-08-07 08:53:41,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1691398420862.9ca78a73cef89f541dcf91876dbfafd9. 2023-08-07 08:53:41,790 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionNoStoreFile/9ca78a73cef89f541dcf91876dbfafd9/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-08-07 08:53:41,790 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-07 08:53:41,790 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1691398420862.9ca78a73cef89f541dcf91876dbfafd9. 2023-08-07 08:53:41,790 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 9ca78a73cef89f541dcf91876dbfafd9: 2023-08-07 08:53:41,792 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 9ca78a73cef89f541dcf91876dbfafd9 2023-08-07 08:53:41,792 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 5b3565e5e309ccbd325fa8dc34e561c4 2023-08-07 08:53:41,793 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 5b3565e5e309ccbd325fa8dc34e561c4, disabling compactions & flushes 2023-08-07 08:53:41,793 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,41,1691398420862.5b3565e5e309ccbd325fa8dc34e561c4. 2023-08-07 08:53:41,793 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1691398420862.5b3565e5e309ccbd325fa8dc34e561c4. 2023-08-07 08:53:41,793 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,41,1691398420862.5b3565e5e309ccbd325fa8dc34e561c4. after waiting 0 ms 2023-08-07 08:53:41,793 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,41,1691398420862.5b3565e5e309ccbd325fa8dc34e561c4. 2023-08-07 08:53:41,793 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=9ca78a73cef89f541dcf91876dbfafd9, regionState=CLOSED 2023-08-07 08:53:41,793 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1691398420862.9ca78a73cef89f541dcf91876dbfafd9.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1691398421793"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398421793"}]},"ts":"1691398421793"} 2023-08-07 08:53:41,797 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionNoStoreFile/5b3565e5e309ccbd325fa8dc34e561c4/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-08-07 08:53:41,797 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=64 2023-08-07 08:53:41,797 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure 9ca78a73cef89f541dcf91876dbfafd9, server=jenkins-hbase4.apache.org,34559,1691398396467 in 158 msec 2023-08-07 08:53:41,798 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-07 08:53:41,798 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,41,1691398420862.5b3565e5e309ccbd325fa8dc34e561c4. 2023-08-07 08:53:41,798 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 5b3565e5e309ccbd325fa8dc34e561c4: 2023-08-07 08:53:41,799 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=9ca78a73cef89f541dcf91876dbfafd9, UNASSIGN in 166 msec 2023-08-07 08:53:41,799 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 5b3565e5e309ccbd325fa8dc34e561c4 2023-08-07 08:53:41,800 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=5b3565e5e309ccbd325fa8dc34e561c4, regionState=CLOSED 2023-08-07 08:53:41,800 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1691398420862.5b3565e5e309ccbd325fa8dc34e561c4.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1691398421800"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398421800"}]},"ts":"1691398421800"} 2023-08-07 08:53:41,804 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-08-07 08:53:41,804 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure 5b3565e5e309ccbd325fa8dc34e561c4, server=jenkins-hbase4.apache.org,34559,1691398396467 in 170 msec 2023-08-07 08:53:41,806 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=65, resume processing ppid=63 2023-08-07 08:53:41,806 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=5b3565e5e309ccbd325fa8dc34e561c4, UNASSIGN in 174 msec 2023-08-07 08:53:41,807 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398421807"}]},"ts":"1691398421807"} 2023-08-07 08:53:41,808 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-08-07 08:53:41,809 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-08-07 08:53:41,811 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 187 msec 2023-08-07 08:53:41,928 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=63 2023-08-07 08:53:41,928 INFO [Listener at localhost/33561] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-08-07 08:53:41,929 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionNoStoreFile 2023-08-07 08:53:41,930 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:53:41,932 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:53:41,932 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:53:41,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=68 2023-08-07 08:53:41,938 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionNoStoreFile/fd6d700fb2bf2b02eb0b469b2b0d401b 2023-08-07 08:53:41,938 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionNoStoreFile/5b3565e5e309ccbd325fa8dc34e561c4 2023-08-07 08:53:41,938 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionNoStoreFile/9ca78a73cef89f541dcf91876dbfafd9 2023-08-07 08:53:41,940 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionNoStoreFile/5b3565e5e309ccbd325fa8dc34e561c4/cf1, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionNoStoreFile/5b3565e5e309ccbd325fa8dc34e561c4/cf2, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionNoStoreFile/5b3565e5e309ccbd325fa8dc34e561c4/recovered.edits] 2023-08-07 08:53:41,940 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionNoStoreFile/9ca78a73cef89f541dcf91876dbfafd9/cf1, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionNoStoreFile/9ca78a73cef89f541dcf91876dbfafd9/cf2, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionNoStoreFile/9ca78a73cef89f541dcf91876dbfafd9/recovered.edits] 2023-08-07 08:53:41,941 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionNoStoreFile/fd6d700fb2bf2b02eb0b469b2b0d401b/cf1, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionNoStoreFile/fd6d700fb2bf2b02eb0b469b2b0d401b/cf2, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionNoStoreFile/fd6d700fb2bf2b02eb0b469b2b0d401b/recovered.edits] 2023-08-07 08:53:41,949 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionNoStoreFile/5b3565e5e309ccbd325fa8dc34e561c4/recovered.edits/7.seqid to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionNoStoreFile/5b3565e5e309ccbd325fa8dc34e561c4/recovered.edits/7.seqid 2023-08-07 08:53:41,949 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionNoStoreFile/9ca78a73cef89f541dcf91876dbfafd9/recovered.edits/7.seqid to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionNoStoreFile/9ca78a73cef89f541dcf91876dbfafd9/recovered.edits/7.seqid 2023-08-07 08:53:41,950 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionNoStoreFile/9ca78a73cef89f541dcf91876dbfafd9 2023-08-07 08:53:41,950 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionNoStoreFile/5b3565e5e309ccbd325fa8dc34e561c4 2023-08-07 08:53:41,950 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionNoStoreFile/fd6d700fb2bf2b02eb0b469b2b0d401b/recovered.edits/4.seqid to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionNoStoreFile/fd6d700fb2bf2b02eb0b469b2b0d401b/recovered.edits/4.seqid 2023-08-07 08:53:41,951 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionNoStoreFile/fd6d700fb2bf2b02eb0b469b2b0d401b 2023-08-07 08:53:41,951 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-08-07 08:53:41,954 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:53:41,955 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-08-07 08:53:41,957 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-08-07 08:53:41,958 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:53:41,958 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-08-07 08:53:41,959 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1691398420360.fd6d700fb2bf2b02eb0b469b2b0d401b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691398421958"}]},"ts":"9223372036854775807"} 2023-08-07 08:53:41,959 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1691398420862.9ca78a73cef89f541dcf91876dbfafd9.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691398421958"}]},"ts":"9223372036854775807"} 2023-08-07 08:53:41,959 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1691398420862.5b3565e5e309ccbd325fa8dc34e561c4.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691398421958"}]},"ts":"9223372036854775807"} 2023-08-07 08:53:41,960 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-08-07 08:53:41,960 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => fd6d700fb2bf2b02eb0b469b2b0d401b, NAME => 'testSplitTableRegionNoStoreFile,,1691398420360.fd6d700fb2bf2b02eb0b469b2b0d401b.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 9ca78a73cef89f541dcf91876dbfafd9, NAME => 'testSplitTableRegionNoStoreFile,,1691398420862.9ca78a73cef89f541dcf91876dbfafd9.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 5b3565e5e309ccbd325fa8dc34e561c4, NAME => 'testSplitTableRegionNoStoreFile,41,1691398420862.5b3565e5e309ccbd325fa8dc34e561c4.', STARTKEY => '41', ENDKEY => ''}] 2023-08-07 08:53:41,960 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-08-07 08:53:41,961 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1691398421961"}]},"ts":"9223372036854775807"} 2023-08-07 08:53:41,962 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-08-07 08:53:41,964 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:53:41,965 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 35 msec 2023-08-07 08:53:42,033 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=68 2023-08-07 08:53:42,034 INFO [Listener at localhost/33561] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-08-07 08:53:42,048 INFO [Listener at localhost/33561] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=401 (was 398) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-996070550_17 at /127.0.0.1:56852 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x36ae5be-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x36ae5be-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x36ae5be-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-996070550_17 at /127.0.0.1:56870 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x36ae5be-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=855 (was 857), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=368 (was 368), ProcessCount=174 (was 174), AvailableMemoryMB=1921 (was 1922) 2023-08-07 08:53:42,065 INFO [Listener at localhost/33561] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=401, OpenFileDescriptor=855, MaxFileDescriptor=60000, SystemLoadAverage=368, ProcessCount=174, AvailableMemoryMB=1920 2023-08-07 08:53:42,065 WARN [Listener at localhost/33561] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-07 08:53:42,067 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-08-07 08:53:42,068 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-08-07 08:53:42,069 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(1029): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,071 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-08-07 08:53:42,071 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-07 08:53:42,073 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:42,074 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da empty. 2023-08-07 08:53:42,074 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:42,074 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-08-07 08:53:42,089 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-08-07 08:53:42,090 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => 0d0fd872f4eb265c927f025c1f8dd4da, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp 2023-08-07 08:53:42,101 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:53:42,101 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1604): Closing 0d0fd872f4eb265c927f025c1f8dd4da, disabling compactions & flushes 2023-08-07 08:53:42,101 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. 2023-08-07 08:53:42,101 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. 2023-08-07 08:53:42,101 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. after waiting 0 ms 2023-08-07 08:53:42,101 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. 2023-08-07 08:53:42,101 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. 2023-08-07 08:53:42,101 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1558): Region close journal for 0d0fd872f4eb265c927f025c1f8dd4da: 2023-08-07 08:53:42,103 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-08-07 08:53:42,104 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1691398422104"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398422104"}]},"ts":"1691398422104"} 2023-08-07 08:53:42,105 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-07 08:53:42,106 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-07 08:53:42,106 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398422106"}]},"ts":"1691398422106"} 2023-08-07 08:53:42,107 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-08-07 08:53:42,111 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-07 08:53:42,112 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-07 08:53:42,112 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-07 08:53:42,112 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-07 08:53:42,112 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-07 08:53:42,112 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=0d0fd872f4eb265c927f025c1f8dd4da, ASSIGN}] 2023-08-07 08:53:42,113 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=0d0fd872f4eb265c927f025c1f8dd4da, ASSIGN 2023-08-07 08:53:42,114 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=0d0fd872f4eb265c927f025c1f8dd4da, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34559,1691398396467; forceNewPlan=false, retain=false 2023-08-07 08:53:42,264 INFO [jenkins-hbase4:33357] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-07 08:53:42,265 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=0d0fd872f4eb265c927f025c1f8dd4da, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,266 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1691398422265"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398422265"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398422265"}]},"ts":"1691398422265"} 2023-08-07 08:53:42,268 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure 0d0fd872f4eb265c927f025c1f8dd4da, server=jenkins-hbase4.apache.org,34559,1691398396467}] 2023-08-07 08:53:42,423 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. 2023-08-07 08:53:42,423 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 0d0fd872f4eb265c927f025c1f8dd4da, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da.', STARTKEY => '', ENDKEY => ''} 2023-08-07 08:53:42,423 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-07 08:53:42,423 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:42,423 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:53:42,423 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:42,423 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:42,424 INFO [StoreOpener-0d0fd872f4eb265c927f025c1f8dd4da-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:42,426 DEBUG [StoreOpener-0d0fd872f4eb265c927f025c1f8dd4da-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1 2023-08-07 08:53:42,426 DEBUG [StoreOpener-0d0fd872f4eb265c927f025c1f8dd4da-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1 2023-08-07 08:53:42,426 INFO [StoreOpener-0d0fd872f4eb265c927f025c1f8dd4da-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0d0fd872f4eb265c927f025c1f8dd4da columnFamilyName cf1 2023-08-07 08:53:42,427 INFO [StoreOpener-0d0fd872f4eb265c927f025c1f8dd4da-1] regionserver.HStore(310): Store=0d0fd872f4eb265c927f025c1f8dd4da/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:42,427 INFO [StoreOpener-0d0fd872f4eb265c927f025c1f8dd4da-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:42,428 DEBUG [StoreOpener-0d0fd872f4eb265c927f025c1f8dd4da-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2 2023-08-07 08:53:42,428 DEBUG [StoreOpener-0d0fd872f4eb265c927f025c1f8dd4da-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2 2023-08-07 08:53:42,429 INFO [StoreOpener-0d0fd872f4eb265c927f025c1f8dd4da-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0d0fd872f4eb265c927f025c1f8dd4da columnFamilyName cf2 2023-08-07 08:53:42,429 INFO [StoreOpener-0d0fd872f4eb265c927f025c1f8dd4da-1] regionserver.HStore(310): Store=0d0fd872f4eb265c927f025c1f8dd4da/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:42,430 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:42,430 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:42,433 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-07 08:53:42,434 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:42,436 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 08:53:42,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 0d0fd872f4eb265c927f025c1f8dd4da; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71007338, jitterRate=0.05809178948402405}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-07 08:53:42,437 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 0d0fd872f4eb265c927f025c1f8dd4da: 2023-08-07 08:53:42,438 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da., pid=71, masterSystemTime=1691398422420 2023-08-07 08:53:42,441 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. 2023-08-07 08:53:42,441 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. 2023-08-07 08:53:42,441 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=0d0fd872f4eb265c927f025c1f8dd4da, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,441 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1691398422441"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691398422441"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691398422441"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398422441"}]},"ts":"1691398422441"} 2023-08-07 08:53:42,444 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=71, resume processing ppid=70 2023-08-07 08:53:42,444 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure 0d0fd872f4eb265c927f025c1f8dd4da, server=jenkins-hbase4.apache.org,34559,1691398396467 in 175 msec 2023-08-07 08:53:42,446 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-08-07 08:53:42,446 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=0d0fd872f4eb265c927f025c1f8dd4da, ASSIGN in 333 msec 2023-08-07 08:53:42,447 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-07 08:53:42,447 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398422447"}]},"ts":"1691398422447"} 2023-08-07 08:53:42,448 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-08-07 08:53:42,451 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-08-07 08:53:42,452 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 383 msec 2023-08-07 08:53:42,577 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:53:42,579 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,579 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-08-07 08:53:42,579 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:42,579 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-08-07 08:53:42,579 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-08-07 08:53:42,579 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,579 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,581 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:42,581 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,581 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:42,581 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:42,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:42,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:42,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:42,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:42,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:42,581 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,581 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:53:42,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:42,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,582 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,582 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,582 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:42,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:42,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:42,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,582 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,582 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,582 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,582 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,582 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,582 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-07 08:53:42,583 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-07 08:53:42,583 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,583 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:42,583 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-07 08:53:42,583 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:42,583 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:42,583 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-07 08:53:42,583 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-07 08:53:42,583 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-07 08:53:42,583 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-07 08:53:42,583 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-07 08:53:42,583 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-07 08:53:42,583 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-07 08:53:42,583 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:53:42,583 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:53:42,583 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. 2023-08-07 08:53:42,583 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. started... 2023-08-07 08:53:42,584 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0d0fd872f4eb265c927f025c1f8dd4da 2/2 column families, dataSize=120 B heapSize=928 B 2023-08-07 08:53:42,586 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,586 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,587 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,587 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:42,587 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,587 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:42,599 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf1/3a64bb0e4a9e41339304ef7ecb5b5d7a 2023-08-07 08:53:42,613 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf2/68bc7551705944cb8c4eaeb76c784f25 2023-08-07 08:53:42,617 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf1/3a64bb0e4a9e41339304ef7ecb5b5d7a as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/3a64bb0e4a9e41339304ef7ecb5b5d7a 2023-08-07 08:53:42,622 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/3a64bb0e4a9e41339304ef7ecb5b5d7a, entries=2, sequenceid=6, filesize=4.8 K 2023-08-07 08:53:42,622 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf2/68bc7551705944cb8c4eaeb76c784f25 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/68bc7551705944cb8c4eaeb76c784f25 2023-08-07 08:53:42,626 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/68bc7551705944cb8c4eaeb76c784f25, entries=2, sequenceid=6, filesize=4.8 K 2023-08-07 08:53:42,627 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 0d0fd872f4eb265c927f025c1f8dd4da in 43ms, sequenceid=6, compaction requested=false 2023-08-07 08:53:42,627 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-08-07 08:53:42,627 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0d0fd872f4eb265c927f025c1f8dd4da: 2023-08-07 08:53:42,627 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. 2023-08-07 08:53:42,628 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:53:42,628 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:53:42,628 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,628 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-07 08:53:42,628 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-07 08:53:42,629 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,629 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:42,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:42,630 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,630 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:42,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:42,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:42,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:42,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:42,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:42,632 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-08-07 08:53:42,632 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7dadfb8[Count = 0] remaining members to acquire global barrier 2023-08-07 08:53:42,632 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-08-07 08:53:42,632 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,633 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,633 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,633 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,633 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-07 08:53:42,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,634 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-07 08:53:42,634 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-07 08:53:42,634 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-07 08:53:42,634 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:53:42,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,634 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,634 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:53:42,634 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:53:42,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,634 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-07 08:53:42,634 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-07 08:53:42,634 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:53:42,635 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-07 08:53:42,635 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:42,635 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-07 08:53:42,641 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-07 08:53:42,641 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:42,641 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-07 08:53:42,642 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,642 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-07 08:53:42,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:42,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:42,642 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:42,642 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-07 08:53:42,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:42,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:42,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:42,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:42,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:42,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:42,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:42,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,34559,1691398396467': 2023-08-07 08:53:42,645 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-08-07 08:53:42,645 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-08-07 08:53:42,645 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:53:42,645 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:53:42,645 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,645 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:53:42,647 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,647 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,647 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:42,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:42,647 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,647 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:42,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,647 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:42,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,647 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:42,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:42,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:42,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:42,648 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:42,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:42,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:42,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:42,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:42,648 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,648 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:42,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:42,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:42,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:42,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:42,650 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:42,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,650 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:42,651 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:42,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:42,663 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:42,663 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,663 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:42,663 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:42,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:42,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:42,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:42,663 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,663 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:42,663 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:42,663 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-08-07 08:53:42,664 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:53:42,663 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:42,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:42,663 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:42,663 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:42,664 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,663 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:42,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:42,664 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-08-07 08:53:42,664 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:53:42,664 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:42,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:42,664 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:42,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:42,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:42,664 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:42,664 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:42,665 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,665 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:42,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:42,764 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:53:42,765 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:53:42,778 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:53:42,780 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-08-07 08:53:42,780 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,780 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-08-07 08:53:42,780 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:42,780 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-08-07 08:53:42,780 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-08-07 08:53:42,781 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,781 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,782 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,782 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:42,782 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:42,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:42,782 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:42,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:42,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:42,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:42,783 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,783 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:53:42,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:42,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:42,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,783 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,783 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,784 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:42,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:42,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,784 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,784 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:42,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,784 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-07 08:53:42,784 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,784 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-07 08:53:42,784 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,784 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,784 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-07 08:53:42,785 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-07 08:53:42,785 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:42,785 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:42,785 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-07 08:53:42,785 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-07 08:53:42,785 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-07 08:53:42,785 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-07 08:53:42,785 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-07 08:53:42,785 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-07 08:53:42,786 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,787 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,787 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,787 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:42,788 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,788 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:42,790 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-07 08:53:42,790 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,791 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-07 08:53:42,791 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:42,791 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-07 08:53:42,793 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:53:42,793 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. 2023-08-07 08:53:42,793 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:53:42,793 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. started... 2023-08-07 08:53:42,793 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0d0fd872f4eb265c927f025c1f8dd4da 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:53:42,806 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf1/ed4b0c71cd7d4265b3fdd040094d678d 2023-08-07 08:53:42,830 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf2/4f5f12f2155941219ae22b6ac5110593 2023-08-07 08:53:42,835 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf1/ed4b0c71cd7d4265b3fdd040094d678d as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/ed4b0c71cd7d4265b3fdd040094d678d 2023-08-07 08:53:42,840 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/ed4b0c71cd7d4265b3fdd040094d678d, entries=10, sequenceid=19, filesize=5.0 K 2023-08-07 08:53:42,841 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf2/4f5f12f2155941219ae22b6ac5110593 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/4f5f12f2155941219ae22b6ac5110593 2023-08-07 08:53:42,845 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/4f5f12f2155941219ae22b6ac5110593, entries=10, sequenceid=19, filesize=5.0 K 2023-08-07 08:53:42,846 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0d0fd872f4eb265c927f025c1f8dd4da in 53ms, sequenceid=19, compaction requested=false 2023-08-07 08:53:42,847 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0d0fd872f4eb265c927f025c1f8dd4da: 2023-08-07 08:53:42,847 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. 2023-08-07 08:53:42,847 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:53:42,847 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:53:42,847 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,847 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-07 08:53:42,847 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-07 08:53:42,850 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,850 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:42,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:42,850 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,850 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:42,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:42,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:42,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:42,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:42,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:42,852 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-08-07 08:53:42,853 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@20cb1251[Count = 0] remaining members to acquire global barrier 2023-08-07 08:53:42,853 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-08-07 08:53:42,853 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,854 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,854 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,854 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,854 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-07 08:53:42,854 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-07 08:53:42,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,854 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-07 08:53:42,854 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-07 08:53:42,854 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:53:42,854 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,854 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:53:42,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,854 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:53:42,855 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-07 08:53:42,855 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-07 08:53:42,855 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:53:42,856 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-07 08:53:42,856 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:42,856 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-07 08:53:42,857 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-07 08:53:42,857 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:42,857 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-07 08:53:42,858 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,858 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-07 08:53:42,859 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:42,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:42,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:42,859 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-07 08:53:42,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:42,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:42,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:42,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:42,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:42,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:42,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:42,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,34559,1691398396467': 2023-08-07 08:53:42,862 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-08-07 08:53:42,862 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-08-07 08:53:42,862 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:53:42,862 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:53:42,862 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,862 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:53:42,864 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,864 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,864 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,864 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:42,864 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:42,864 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,864 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:42,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:42,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:42,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:42,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:42,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,864 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:42,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,865 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:42,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:42,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:42,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:42,866 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:42,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:42,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:42,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:42,868 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:42,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,868 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:42,868 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:42,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:42,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:42,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:42,871 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:42,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:42,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:42,872 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:42,871 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:42,872 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:42,872 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:42,871 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:42,872 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,872 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:42,872 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:42,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:42,872 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-08-07 08:53:42,872 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:53:42,872 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:42,872 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,872 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:42,872 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,872 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,872 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:42,872 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,872 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:42,872 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:42,872 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-08-07 08:53:42,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:42,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:42,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:42,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:42,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:42,873 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:53:42,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:42,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:42,973 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:53:42,974 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:53:42,987 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:53:42,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-08-07 08:53:42,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,989 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-08-07 08:53:42,989 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:42,989 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-08-07 08:53:42,989 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-08-07 08:53:42,989 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,989 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,990 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,991 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:42,991 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:42,991 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:42,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:42,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:42,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:42,991 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:42,991 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:53:42,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:42,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:42,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:42,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,991 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,992 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,992 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:42,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,992 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:42,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,992 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:42,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,992 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-07 08:53:42,992 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,992 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,992 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-07 08:53:42,992 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:42,992 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-07 08:53:42,992 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-07 08:53:42,992 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,992 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-07 08:53:42,993 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-07 08:53:42,993 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-07 08:53:42,992 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-07 08:53:42,993 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:42,993 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,993 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-07 08:53:42,993 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-07 08:53:42,993 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:42,993 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-07 08:53:42,993 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-07 08:53:42,993 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-07 08:53:42,993 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:53:42,993 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:53:42,993 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. 2023-08-07 08:53:42,993 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. started... 2023-08-07 08:53:42,993 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0d0fd872f4eb265c927f025c1f8dd4da 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:53:42,995 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,995 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,996 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,996 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:42,996 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:42,996 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:43,002 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf1/fe73c4030e5f45e5b2e8db40593817a7 2023-08-07 08:53:43,014 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf2/7228be319c5144239680685192b85fb5 2023-08-07 08:53:43,017 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf1/fe73c4030e5f45e5b2e8db40593817a7 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/fe73c4030e5f45e5b2e8db40593817a7 2023-08-07 08:53:43,022 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/fe73c4030e5f45e5b2e8db40593817a7, entries=10, sequenceid=32, filesize=5.0 K 2023-08-07 08:53:43,023 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf2/7228be319c5144239680685192b85fb5 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/7228be319c5144239680685192b85fb5 2023-08-07 08:53:43,026 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/7228be319c5144239680685192b85fb5, entries=10, sequenceid=32, filesize=5.0 K 2023-08-07 08:53:43,027 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0d0fd872f4eb265c927f025c1f8dd4da in 34ms, sequenceid=32, compaction requested=true 2023-08-07 08:53:43,027 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0d0fd872f4eb265c927f025c1f8dd4da: 2023-08-07 08:53:43,027 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. 2023-08-07 08:53:43,027 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:53:43,027 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:53:43,027 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,027 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-07 08:53:43,027 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-07 08:53:43,032 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,032 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:43,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:43,032 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,032 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:43,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:43,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:43,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:43,034 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-08-07 08:53:43,034 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@52813d48[Count = 0] remaining members to acquire global barrier 2023-08-07 08:53:43,034 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-08-07 08:53:43,034 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,037 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,037 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,037 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,037 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-07 08:53:43,037 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,037 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:53:43,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,037 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-07 08:53:43,037 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-07 08:53:43,037 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:53:43,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,037 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-07 08:53:43,037 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-07 08:53:43,037 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-07 08:53:43,037 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:53:43,037 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:53:43,039 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,039 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-07 08:53:43,039 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:43,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:43,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:43,039 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-07 08:53:43,039 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-07 08:53:43,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:43,040 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:43,039 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-07 08:53:43,040 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:43,040 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-07 08:53:43,040 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-07 08:53:43,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:43,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:43,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,34559,1691398396467': 2023-08-07 08:53:43,042 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-08-07 08:53:43,042 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-08-07 08:53:43,042 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:53:43,042 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:53:43,042 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,042 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:53:43,044 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,044 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,044 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,044 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:43,044 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,044 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:43,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:43,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:43,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,044 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,044 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:43,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:43,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:43,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:43,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:43,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:43,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:43,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:43,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,045 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,046 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:43,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,047 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:43,048 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,048 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,052 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,052 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,052 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:43,052 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,052 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:43,052 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,052 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:43,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:43,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:43,052 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:43,052 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:43,052 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-08-07 08:53:43,052 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:53:43,052 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:43,052 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,052 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:43,052 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:43,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:43,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:43,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:43,052 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:43,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:43,053 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-08-07 08:53:43,052 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:43,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:43,053 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,053 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:53:43,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:43,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:43,053 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,053 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,153 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:53:43,154 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:53:43,169 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:53:43,170 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-08-07 08:53:43,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,171 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-08-07 08:53:43,171 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:43,171 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-08-07 08:53:43,171 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-08-07 08:53:43,171 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,171 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,173 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:43,173 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,173 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:43,173 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:43,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:43,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:43,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:43,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:43,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:43,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:43,173 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,173 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:53:43,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,174 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,174 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,174 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:43,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,175 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:43,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:43,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,175 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,175 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-07 08:53:43,175 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,175 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,175 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-07 08:53:43,175 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-07 08:53:43,175 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,175 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-07 08:53:43,175 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:43,175 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-07 08:53:43,175 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:43,175 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-07 08:53:43,175 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-07 08:53:43,175 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-07 08:53:43,175 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-07 08:53:43,175 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-07 08:53:43,175 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-07 08:53:43,176 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,176 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-07 08:53:43,176 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:43,176 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-07 08:53:43,176 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:53:43,176 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:53:43,176 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. 2023-08-07 08:53:43,176 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. started... 2023-08-07 08:53:43,176 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0d0fd872f4eb265c927f025c1f8dd4da 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:53:43,177 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,177 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,178 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,178 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,178 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:43,178 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:43,192 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf1/d9bc9e401b7f42a2a6936beaf6b25a52 2023-08-07 08:53:43,203 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf2/39fc693feb564d94a22dd0a477a1d4f1 2023-08-07 08:53:43,208 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf1/d9bc9e401b7f42a2a6936beaf6b25a52 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/d9bc9e401b7f42a2a6936beaf6b25a52 2023-08-07 08:53:43,211 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/d9bc9e401b7f42a2a6936beaf6b25a52, entries=10, sequenceid=45, filesize=5.0 K 2023-08-07 08:53:43,212 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf2/39fc693feb564d94a22dd0a477a1d4f1 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/39fc693feb564d94a22dd0a477a1d4f1 2023-08-07 08:53:43,215 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/39fc693feb564d94a22dd0a477a1d4f1, entries=10, sequenceid=45, filesize=5.0 K 2023-08-07 08:53:43,216 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0d0fd872f4eb265c927f025c1f8dd4da in 40ms, sequenceid=45, compaction requested=true 2023-08-07 08:53:43,216 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0d0fd872f4eb265c927f025c1f8dd4da: 2023-08-07 08:53:43,216 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. 2023-08-07 08:53:43,216 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:53:43,216 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:53:43,216 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,216 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-07 08:53:43,216 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-07 08:53:43,218 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,218 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:43,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:43,218 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,218 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:43,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:43,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:43,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:43,220 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-08-07 08:53:43,220 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4661475f[Count = 0] remaining members to acquire global barrier 2023-08-07 08:53:43,220 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-08-07 08:53:43,220 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,222 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,222 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,222 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,222 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-07 08:53:43,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,223 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,223 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:53:43,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,223 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-07 08:53:43,223 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-07 08:53:43,223 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-07 08:53:43,223 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:53:43,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,223 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:53:43,223 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-07 08:53:43,223 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-07 08:53:43,223 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:53:43,224 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-07 08:53:43,224 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:43,224 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-07 08:53:43,225 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-07 08:53:43,225 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:43,225 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,225 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-07 08:53:43,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:43,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:43,225 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-07 08:53:43,225 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:43,225 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-07 08:53:43,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:43,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:43,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:43,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,34559,1691398396467': 2023-08-07 08:53:43,228 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-08-07 08:53:43,228 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-08-07 08:53:43,228 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:53:43,228 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:53:43,228 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,228 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:53:43,234 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,234 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,234 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:43,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,234 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:43,234 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,234 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:43,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:43,234 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:43,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,234 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:43,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:43,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:43,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:43,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:43,235 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:43,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:43,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,235 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:43,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,237 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:43,237 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,238 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,240 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,240 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,240 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,240 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:43,240 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:43,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:43,241 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:43,240 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:43,240 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:43,241 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:43,241 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-08-07 08:53:43,241 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:53:43,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:43,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:43,240 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:43,241 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-08-07 08:53:43,240 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:43,241 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:53:43,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:43,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:43,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:43,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:43,241 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,241 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,241 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,241 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,242 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,242 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,242 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:43,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:43,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:43,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:43,341 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:53:43,342 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:53:43,354 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:53:43,356 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-08-07 08:53:43,356 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,356 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-08-07 08:53:43,356 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:43,356 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-08-07 08:53:43,356 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-08-07 08:53:43,357 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,357 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,358 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:43,358 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:43,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:43,358 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:43,358 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:43,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:43,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:43,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:43,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:43,359 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,359 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:53:43,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,359 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,359 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:43,359 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,359 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:43,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,360 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-07 08:53:43,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:43,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,360 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,360 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,360 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-07 08:53:43,360 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:43,360 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,360 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-07 08:53:43,360 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-07 08:53:43,360 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-07 08:53:43,360 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-07 08:53:43,360 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-07 08:53:43,360 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,360 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,360 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-07 08:53:43,360 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:43,360 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-07 08:53:43,360 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:43,360 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-07 08:53:43,361 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-07 08:53:43,361 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-07 08:53:43,361 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-07 08:53:43,361 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:53:43,361 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:53:43,361 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. 2023-08-07 08:53:43,361 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. started... 2023-08-07 08:53:43,361 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0d0fd872f4eb265c927f025c1f8dd4da 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:53:43,362 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,363 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,363 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,363 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:43,364 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,364 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:43,370 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf1/a65272dd6d024b02b2d8eef892774f87 2023-08-07 08:53:43,385 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf2/93712aa9572140b1b56e5833f35671e6 2023-08-07 08:53:43,390 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf1/a65272dd6d024b02b2d8eef892774f87 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/a65272dd6d024b02b2d8eef892774f87 2023-08-07 08:53:43,394 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/a65272dd6d024b02b2d8eef892774f87, entries=10, sequenceid=58, filesize=5.0 K 2023-08-07 08:53:43,394 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf2/93712aa9572140b1b56e5833f35671e6 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/93712aa9572140b1b56e5833f35671e6 2023-08-07 08:53:43,399 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/93712aa9572140b1b56e5833f35671e6, entries=10, sequenceid=58, filesize=5.0 K 2023-08-07 08:53:43,399 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0d0fd872f4eb265c927f025c1f8dd4da in 38ms, sequenceid=58, compaction requested=true 2023-08-07 08:53:43,400 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0d0fd872f4eb265c927f025c1f8dd4da: 2023-08-07 08:53:43,400 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. 2023-08-07 08:53:43,400 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:53:43,400 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:53:43,400 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,400 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-07 08:53:43,400 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-07 08:53:43,401 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,402 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:43,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:43,402 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,402 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:43,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:43,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:43,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:43,404 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-08-07 08:53:43,404 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@17176734[Count = 0] remaining members to acquire global barrier 2023-08-07 08:53:43,404 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-08-07 08:53:43,404 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,406 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,406 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,406 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,406 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-07 08:53:43,406 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-07 08:53:43,406 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,406 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:53:43,406 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-07 08:53:43,406 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-07 08:53:43,406 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:53:43,406 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:53:43,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,407 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-07 08:53:43,407 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-07 08:53:43,407 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:53:43,408 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-07 08:53:43,408 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:43,408 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-07 08:53:43,408 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:43,408 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-07 08:53:43,408 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-07 08:53:43,409 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,409 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-07 08:53:43,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:43,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:43,409 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:43,410 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-07 08:53:43,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:43,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:43,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:43,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,34559,1691398396467': 2023-08-07 08:53:43,413 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-08-07 08:53:43,413 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-08-07 08:53:43,413 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:53:43,413 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:53:43,413 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,413 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:53:43,415 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,415 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,415 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,415 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,415 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,415 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:43,415 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,415 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:43,415 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,415 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:43,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:43,416 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,415 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:43,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:43,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:43,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:43,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:43,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:43,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:43,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:43,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,416 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,417 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:43,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,419 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:43,419 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,419 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,422 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:43,422 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:43,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:43,422 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:43,422 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:43,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:43,422 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:43,422 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:43,423 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:43,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:43,422 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,423 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,423 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,423 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,423 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,423 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,423 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,423 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,423 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,423 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:43,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:43,422 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:43,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:43,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:43,423 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-08-07 08:53:43,423 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:53:43,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:43,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:43,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:43,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:43,423 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-08-07 08:53:43,424 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:53:43,524 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:53:43,525 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:53:43,536 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:53:43,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-08-07 08:53:43,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,538 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-08-07 08:53:43,538 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:43,538 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-08-07 08:53:43,538 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-08-07 08:53:43,539 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,539 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,544 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:43,544 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:43,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:43,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:43,544 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,544 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:43,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:43,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:43,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:43,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:43,544 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,544 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:53:43,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,545 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,545 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,545 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:43,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:43,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:43,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,545 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,545 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,545 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,545 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-07 08:53:43,545 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,545 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-07 08:53:43,546 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-07 08:53:43,546 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,546 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-07 08:53:43,546 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-07 08:53:43,546 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:43,546 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,546 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-07 08:53:43,546 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-07 08:53:43,546 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:43,546 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-07 08:53:43,546 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-07 08:53:43,546 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:43,546 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-07 08:53:43,546 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-07 08:53:43,546 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-07 08:53:43,546 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-07 08:53:43,546 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:53:43,546 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:53:43,546 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. 2023-08-07 08:53:43,546 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. started... 2023-08-07 08:53:43,546 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0d0fd872f4eb265c927f025c1f8dd4da 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:53:43,547 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,548 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,548 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,548 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:43,548 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,548 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:43,557 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf1/75e432bba3d54ef486faa042619b92f8 2023-08-07 08:53:43,569 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf2/690b4cd18b0d47bea9d964a7c786568b 2023-08-07 08:53:43,573 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf1/75e432bba3d54ef486faa042619b92f8 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/75e432bba3d54ef486faa042619b92f8 2023-08-07 08:53:43,577 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/75e432bba3d54ef486faa042619b92f8, entries=10, sequenceid=71, filesize=5.0 K 2023-08-07 08:53:43,578 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf2/690b4cd18b0d47bea9d964a7c786568b as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/690b4cd18b0d47bea9d964a7c786568b 2023-08-07 08:53:43,582 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/690b4cd18b0d47bea9d964a7c786568b, entries=10, sequenceid=71, filesize=5.0 K 2023-08-07 08:53:43,582 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0d0fd872f4eb265c927f025c1f8dd4da in 36ms, sequenceid=71, compaction requested=true 2023-08-07 08:53:43,583 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0d0fd872f4eb265c927f025c1f8dd4da: 2023-08-07 08:53:43,583 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. 2023-08-07 08:53:43,583 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:53:43,583 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:53:43,583 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,583 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-07 08:53:43,583 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-07 08:53:43,584 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,584 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:43,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:43,585 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,585 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:43,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:43,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:43,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:43,586 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-08-07 08:53:43,586 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6fb6b61d[Count = 0] remaining members to acquire global barrier 2023-08-07 08:53:43,586 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-08-07 08:53:43,587 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,589 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,589 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,589 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,589 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-07 08:53:43,589 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-07 08:53:43,589 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-07 08:53:43,589 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,589 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:53:43,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,589 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:53:43,589 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-07 08:53:43,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,589 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:53:43,589 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-07 08:53:43,589 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-07 08:53:43,589 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:53:43,591 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-07 08:53:43,591 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:43,591 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-07 08:53:43,591 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-07 08:53:43,591 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-07 08:53:43,591 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,591 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:43,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:43,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:43,591 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:43,591 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-07 08:53:43,592 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-07 08:53:43,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:43,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:43,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:43,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,34559,1691398396467': 2023-08-07 08:53:43,595 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-08-07 08:53:43,595 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-08-07 08:53:43,595 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:53:43,595 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:53:43,595 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,595 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:53:43,597 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,597 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,597 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,597 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:43,597 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:43,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,597 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:43,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:43,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,597 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:43,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:43,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:43,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:43,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:43,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:43,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:43,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:43,598 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:43,598 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,599 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:43,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,600 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,600 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,601 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,604 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,604 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,604 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:43,605 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:43,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:43,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:43,605 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:43,605 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:43,605 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:43,605 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:43,605 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,605 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:43,605 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:43,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:43,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:43,605 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-08-07 08:53:43,605 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:53:43,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:43,605 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:43,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:43,605 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:43,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:43,605 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,605 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-08-07 08:53:43,606 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,606 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:53:43,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:43,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:43,606 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,606 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,606 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,706 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:53:43,707 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:53:43,719 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:53:43,721 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-08-07 08:53:43,721 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,721 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-08-07 08:53:43,721 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:43,721 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-08-07 08:53:43,721 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-08-07 08:53:43,722 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,722 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,723 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:43,723 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:43,723 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:43,723 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:43,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:43,723 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:43,723 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,723 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:43,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:43,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:43,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,724 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,724 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:53:43,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,724 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,724 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,724 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:43,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,725 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:43,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:43,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,725 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,725 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,725 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-07 08:53:43,726 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-07 08:53:43,726 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-07 08:53:43,726 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,726 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,726 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-07 08:53:43,726 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-07 08:53:43,726 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:43,726 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,726 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-07 08:53:43,726 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-07 08:53:43,726 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-07 08:53:43,726 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-07 08:53:43,726 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:43,726 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:43,726 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-07 08:53:43,726 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-07 08:53:43,726 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-07 08:53:43,727 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-07 08:53:43,727 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:53:43,727 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:53:43,727 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. 2023-08-07 08:53:43,727 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. started... 2023-08-07 08:53:43,728 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0d0fd872f4eb265c927f025c1f8dd4da 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-08-07 08:53:43,728 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,729 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,729 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,729 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:43,729 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,729 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:43,738 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf1/cfa6c2210933401bb7c7f9b187f92401 2023-08-07 08:53:43,742 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cfa6c2210933401bb7c7f9b187f92401 2023-08-07 08:53:43,750 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf2/e0d378647ec74ea1b52947731f7efefb 2023-08-07 08:53:43,754 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e0d378647ec74ea1b52947731f7efefb 2023-08-07 08:53:43,754 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf1/cfa6c2210933401bb7c7f9b187f92401 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/cfa6c2210933401bb7c7f9b187f92401 2023-08-07 08:53:43,758 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cfa6c2210933401bb7c7f9b187f92401 2023-08-07 08:53:43,758 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/cfa6c2210933401bb7c7f9b187f92401, entries=9, sequenceid=83, filesize=5.1 K 2023-08-07 08:53:43,758 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf2/e0d378647ec74ea1b52947731f7efefb as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/e0d378647ec74ea1b52947731f7efefb 2023-08-07 08:53:43,761 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e0d378647ec74ea1b52947731f7efefb 2023-08-07 08:53:43,762 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/e0d378647ec74ea1b52947731f7efefb, entries=9, sequenceid=83, filesize=5.1 K 2023-08-07 08:53:43,762 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for 0d0fd872f4eb265c927f025c1f8dd4da in 35ms, sequenceid=83, compaction requested=true 2023-08-07 08:53:43,762 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0d0fd872f4eb265c927f025c1f8dd4da: 2023-08-07 08:53:43,762 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. 2023-08-07 08:53:43,763 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:53:43,763 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:53:43,763 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,763 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-07 08:53:43,763 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-07 08:53:43,765 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:43,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:43,765 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:43,766 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,766 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:43,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:43,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:43,767 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-08-07 08:53:43,767 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3406adde[Count = 0] remaining members to acquire global barrier 2023-08-07 08:53:43,767 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-08-07 08:53:43,767 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,768 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,768 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,768 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,769 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,769 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:53:43,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,769 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-07 08:53:43,769 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-07 08:53:43,769 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-07 08:53:43,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,769 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:53:43,769 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-07 08:53:43,769 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-07 08:53:43,769 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:53:43,769 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-07 08:53:43,769 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:53:43,773 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-07 08:53:43,773 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,773 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-07 08:53:43,773 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:43,773 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-07 08:53:43,773 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-07 08:53:43,773 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:43,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:43,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:43,773 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:43,773 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-07 08:53:43,773 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-07 08:53:43,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:43,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:43,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:43,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,34559,1691398396467': 2023-08-07 08:53:43,777 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-08-07 08:53:43,777 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-08-07 08:53:43,777 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:53:43,777 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:53:43,777 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,777 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:53:43,779 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,779 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,779 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:43,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:43,779 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,780 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:43,779 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:43,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:43,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,780 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:43,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:43,780 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:43,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:43,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:43,781 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:43,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:43,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:43,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,781 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:43,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,783 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,783 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,783 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,785 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:43,785 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:43,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:43,785 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:43,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:43,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:43,785 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:43,785 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:43,785 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,786 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:43,786 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:43,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:43,786 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-08-07 08:53:43,786 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:53:43,786 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:43,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:43,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:43,786 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:43,786 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-08-07 08:53:43,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:43,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:43,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:43,786 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:43,786 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:53:43,786 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,787 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,787 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,787 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,787 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,787 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,787 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,887 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:53:43,888 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:53:43,895 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:53:43,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-08-07 08:53:43,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,897 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-08-07 08:53:43,897 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:43,897 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-08-07 08:53:43,897 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-08-07 08:53:43,897 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,897 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,899 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:43,899 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:43,899 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:43,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:43,899 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:43,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:43,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:43,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:43,899 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,899 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:53:43,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:43,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,900 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,900 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,900 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:43,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,900 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:43,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,900 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:43,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,900 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-07 08:53:43,900 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-07 08:53:43,900 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,900 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,900 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-07 08:53:43,900 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-07 08:53:43,900 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:43,900 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,900 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-07 08:53:43,900 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-07 08:53:43,900 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:43,901 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-07 08:53:43,901 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-07 08:53:43,901 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-07 08:53:43,901 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-07 08:53:43,901 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,901 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-07 08:53:43,901 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-07 08:53:43,901 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:43,901 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-07 08:53:43,901 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:53:43,901 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:53:43,901 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. 2023-08-07 08:53:43,901 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. started... 2023-08-07 08:53:43,901 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0d0fd872f4eb265c927f025c1f8dd4da 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-08-07 08:53:43,902 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,904 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,904 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,904 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:43,904 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,904 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:43,912 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf1/fb927cf3b10e4f6bab7afd25906a1091 2023-08-07 08:53:43,915 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fb927cf3b10e4f6bab7afd25906a1091 2023-08-07 08:53:43,923 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf2/876913088cb546d1b4701088081483d7 2023-08-07 08:53:43,926 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 876913088cb546d1b4701088081483d7 2023-08-07 08:53:43,927 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf1/fb927cf3b10e4f6bab7afd25906a1091 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/fb927cf3b10e4f6bab7afd25906a1091 2023-08-07 08:53:43,930 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fb927cf3b10e4f6bab7afd25906a1091 2023-08-07 08:53:43,930 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/fb927cf3b10e4f6bab7afd25906a1091, entries=5, sequenceid=91, filesize=5.0 K 2023-08-07 08:53:43,931 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf2/876913088cb546d1b4701088081483d7 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/876913088cb546d1b4701088081483d7 2023-08-07 08:53:43,934 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 876913088cb546d1b4701088081483d7 2023-08-07 08:53:43,934 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/876913088cb546d1b4701088081483d7, entries=5, sequenceid=91, filesize=5.0 K 2023-08-07 08:53:43,935 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 0d0fd872f4eb265c927f025c1f8dd4da in 34ms, sequenceid=91, compaction requested=true 2023-08-07 08:53:43,935 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0d0fd872f4eb265c927f025c1f8dd4da: 2023-08-07 08:53:43,935 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. 2023-08-07 08:53:43,935 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:53:43,935 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:53:43,935 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,935 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-07 08:53:43,935 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-07 08:53:43,937 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,937 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:43,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:43,937 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,937 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:43,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:43,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:43,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:43,939 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-08-07 08:53:43,939 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@78915fbe[Count = 0] remaining members to acquire global barrier 2023-08-07 08:53:43,939 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-08-07 08:53:43,939 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,940 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,941 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,941 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,941 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-07 08:53:43,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,941 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-07 08:53:43,941 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-07 08:53:43,941 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,941 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:53:43,941 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:53:43,941 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-07 08:53:43,941 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-07 08:53:43,941 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:53:43,941 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-07 08:53:43,941 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:53:43,943 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-07 08:53:43,943 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-07 08:53:43,943 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:43,943 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:43,943 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,943 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-07 08:53:43,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:43,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:43,943 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-07 08:53:43,943 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-07 08:53:43,943 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:43,943 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-07 08:53:43,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:43,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:43,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:43,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,34559,1691398396467': 2023-08-07 08:53:43,946 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-08-07 08:53:43,946 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-08-07 08:53:43,946 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:53:43,946 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:53:43,946 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,946 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:53:43,949 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,949 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,949 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,949 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:43,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,949 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:43,949 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:43,949 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:43,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:43,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:43,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:43,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:43,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:43,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:43,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:43,949 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:43,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,950 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,950 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:43,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,952 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:43,952 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,952 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,958 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,958 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,958 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:43,958 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:43,958 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:43,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:43,958 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:43,958 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-08-07 08:53:43,958 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:53:43,958 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:43,958 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:43,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:43,958 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:43,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:43,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:43,958 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:43,958 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-08-07 08:53:43,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:43,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:43,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:43,958 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:43,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:43,959 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:53:43,958 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,959 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:43,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:43,959 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:43,959 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:43,959 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:43,959 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:43,959 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,059 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:53:44,060 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:53:44,068 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:53:44,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-08-07 08:53:44,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,069 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-08-07 08:53:44,069 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:44,069 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-08-07 08:53:44,069 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-08-07 08:53:44,070 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,070 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,071 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:44,071 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:44,071 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:44,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:44,071 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:44,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:44,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:44,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:44,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:44,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:44,071 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:44,071 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:53:44,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,072 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,072 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,072 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:44,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:44,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:53:44,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,072 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,073 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,072 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,073 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-07 08:53:44,073 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,073 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-07 08:53:44,073 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:44,073 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-07 08:53:44,073 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-07 08:53:44,073 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,073 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-07 08:53:44,073 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-07 08:53:44,073 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:44,073 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,073 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-07 08:53:44,073 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-07 08:53:44,073 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-07 08:53:44,073 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-07 08:53:44,073 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:53:44,073 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-07 08:53:44,073 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-07 08:53:44,074 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-07 08:53:44,074 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:53:44,074 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:53:44,074 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. 2023-08-07 08:53:44,075 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. started... 2023-08-07 08:53:44,075 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0d0fd872f4eb265c927f025c1f8dd4da 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-08-07 08:53:44,076 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,077 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,077 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,077 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:44,077 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,077 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:44,084 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf1/2f6d4ffab62d47d8b8a61ce088a54c99 2023-08-07 08:53:44,088 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2f6d4ffab62d47d8b8a61ce088a54c99 2023-08-07 08:53:44,095 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf2/48528e10cbed4040851175be536b961e 2023-08-07 08:53:44,099 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 48528e10cbed4040851175be536b961e 2023-08-07 08:53:44,099 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf1/2f6d4ffab62d47d8b8a61ce088a54c99 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/2f6d4ffab62d47d8b8a61ce088a54c99 2023-08-07 08:53:44,104 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2f6d4ffab62d47d8b8a61ce088a54c99 2023-08-07 08:53:44,104 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/2f6d4ffab62d47d8b8a61ce088a54c99, entries=5, sequenceid=99, filesize=5.0 K 2023-08-07 08:53:44,105 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf2/48528e10cbed4040851175be536b961e as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/48528e10cbed4040851175be536b961e 2023-08-07 08:53:44,108 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 48528e10cbed4040851175be536b961e 2023-08-07 08:53:44,108 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/48528e10cbed4040851175be536b961e, entries=5, sequenceid=99, filesize=5.0 K 2023-08-07 08:53:44,109 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 0d0fd872f4eb265c927f025c1f8dd4da in 34ms, sequenceid=99, compaction requested=true 2023-08-07 08:53:44,109 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0d0fd872f4eb265c927f025c1f8dd4da: 2023-08-07 08:53:44,109 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. 2023-08-07 08:53:44,109 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:53:44,109 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:53:44,109 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:44,109 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-07 08:53:44,110 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-07 08:53:44,111 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:44,111 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:44,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:44,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:44,111 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,111 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:53:44,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:44,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:44,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:44,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:44,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:44,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:44,113 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-08-07 08:53:44,113 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5493b95f[Count = 0] remaining members to acquire global barrier 2023-08-07 08:53:44,113 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-08-07 08:53:44,113 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,114 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,114 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,114 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,115 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-07 08:53:44,115 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:44,115 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:53:44,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,115 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-07 08:53:44,115 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:53:44,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,115 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-07 08:53:44,115 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-07 08:53:44,115 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-07 08:53:44,115 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-07 08:53:44,115 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:53:44,115 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:53:44,118 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-07 08:53:44,118 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-07 08:53:44,118 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-07 08:53:44,118 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:44,119 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:44,118 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:44,118 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:44,119 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-07 08:53:44,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:44,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:44,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:44,119 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-07 08:53:44,119 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-07 08:53:44,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:44,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:44,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:44,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:44,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:44,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:44,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:44,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:44,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:44,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,34559,1691398396467': 2023-08-07 08:53:44,122 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-08-07 08:53:44,122 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-08-07 08:53:44,122 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:53:44,122 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:53:44,122 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,122 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:53:44,124 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,124 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,124 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,124 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,124 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:44,124 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:44,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:53:44,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:53:44,124 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:44,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,124 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:44,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:44,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:44,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:44,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:44,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:53:44,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,125 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:44,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:44,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:44,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,125 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:44,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:53:44,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:44,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:44,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:44,127 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:44,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:53:44,127 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:44,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,127 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:44,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:44,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:44,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:44,130 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:44,130 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,130 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:44,130 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:44,130 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:44,130 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:44,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:44,130 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:53:44,130 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:44,130 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:53:44,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:44,130 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-08-07 08:53:44,130 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:53:44,130 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:44,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:44,130 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:44,130 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:53:44,130 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:53:44,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:53:44,131 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-08-07 08:53:44,130 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,131 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:53:44,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:44,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:44,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:53:44,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:44,131 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:44,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:44,131 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:44,131 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,131 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:44,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:53:44,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:53:44,231 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:53:44,232 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:53:44,234 DEBUG [Listener at localhost/33561] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:44,235 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0d0fd872f4eb265c927f025c1f8dd4da, daughterA=890310475de9dc263dd17491ca380a4c, daughterB=246cb6e862f1133bf06f7e32263c91b4 2023-08-07 08:53:44,236 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0d0fd872f4eb265c927f025c1f8dd4da, daughterA=890310475de9dc263dd17491ca380a4c, daughterB=246cb6e862f1133bf06f7e32263c91b4 2023-08-07 08:53:44,236 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0d0fd872f4eb265c927f025c1f8dd4da, daughterA=890310475de9dc263dd17491ca380a4c, daughterB=246cb6e862f1133bf06f7e32263c91b4 2023-08-07 08:53:44,236 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0d0fd872f4eb265c927f025c1f8dd4da, daughterA=890310475de9dc263dd17491ca380a4c, daughterB=246cb6e862f1133bf06f7e32263c91b4 2023-08-07 08:53:44,238 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=0d0fd872f4eb265c927f025c1f8dd4da, UNASSIGN}] 2023-08-07 08:53:44,238 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=0d0fd872f4eb265c927f025c1f8dd4da, UNASSIGN 2023-08-07 08:53:44,239 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=0d0fd872f4eb265c927f025c1f8dd4da, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:44,239 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1691398424239"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398424239"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398424239"}]},"ts":"1691398424239"} 2023-08-07 08:53:44,240 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure 0d0fd872f4eb265c927f025c1f8dd4da, server=jenkins-hbase4.apache.org,34559,1691398396467}] 2023-08-07 08:53:44,392 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,392 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 0d0fd872f4eb265c927f025c1f8dd4da, disabling compactions & flushes 2023-08-07 08:53:44,392 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. 2023-08-07 08:53:44,392 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. 2023-08-07 08:53:44,392 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. after waiting 0 ms 2023-08-07 08:53:44,392 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. 2023-08-07 08:53:44,392 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 0d0fd872f4eb265c927f025c1f8dd4da 2/2 column families, dataSize=50 B heapSize=720 B 2023-08-07 08:53:44,402 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf1/233e2114e35449299d471e817e390162 2023-08-07 08:53:44,405 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 233e2114e35449299d471e817e390162 2023-08-07 08:53:44,414 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf2/1f001470df6748aea71649cffaae8609 2023-08-07 08:53:44,418 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1f001470df6748aea71649cffaae8609 2023-08-07 08:53:44,418 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf1/233e2114e35449299d471e817e390162 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/233e2114e35449299d471e817e390162 2023-08-07 08:53:44,422 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 233e2114e35449299d471e817e390162 2023-08-07 08:53:44,422 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/233e2114e35449299d471e817e390162, entries=1, sequenceid=103, filesize=4.9 K 2023-08-07 08:53:44,423 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.tmp/cf2/1f001470df6748aea71649cffaae8609 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/1f001470df6748aea71649cffaae8609 2023-08-07 08:53:44,426 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1f001470df6748aea71649cffaae8609 2023-08-07 08:53:44,426 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/1f001470df6748aea71649cffaae8609, entries=1, sequenceid=103, filesize=4.9 K 2023-08-07 08:53:44,427 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for 0d0fd872f4eb265c927f025c1f8dd4da in 35ms, sequenceid=103, compaction requested=true 2023-08-07 08:53:44,432 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-08-07 08:53:44,432 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-07 08:53:44,432 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da. 2023-08-07 08:53:44,432 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 0d0fd872f4eb265c927f025c1f8dd4da: 2023-08-07 08:53:44,434 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,434 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=0d0fd872f4eb265c927f025c1f8dd4da, regionState=CLOSED 2023-08-07 08:53:44,434 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1691398424434"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398424434"}]},"ts":"1691398424434"} 2023-08-07 08:53:44,437 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-08-07 08:53:44,437 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure 0d0fd872f4eb265c927f025c1f8dd4da, server=jenkins-hbase4.apache.org,34559,1691398396467 in 195 msec 2023-08-07 08:53:44,438 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-08-07 08:53:44,438 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=0d0fd872f4eb265c927f025c1f8dd4da, UNASSIGN in 199 msec 2023-08-07 08:53:44,455 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=0d0fd872f4eb265c927f025c1f8dd4da, threads=16 2023-08-07 08:53:44,455 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/1f001470df6748aea71649cffaae8609 for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,455 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/39fc693feb564d94a22dd0a477a1d4f1 for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,455 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/48528e10cbed4040851175be536b961e for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,461 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/4f5f12f2155941219ae22b6ac5110593 for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,462 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/690b4cd18b0d47bea9d964a7c786568b for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,462 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/68bc7551705944cb8c4eaeb76c784f25 for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,463 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/7228be319c5144239680685192b85fb5 for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,463 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/876913088cb546d1b4701088081483d7 for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,463 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/93712aa9572140b1b56e5833f35671e6 for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,464 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/e0d378647ec74ea1b52947731f7efefb for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,464 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/233e2114e35449299d471e817e390162 for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,466 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/2f6d4ffab62d47d8b8a61ce088a54c99 for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,467 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/3a64bb0e4a9e41339304ef7ecb5b5d7a for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,467 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/75e432bba3d54ef486faa042619b92f8 for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,471 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/a65272dd6d024b02b2d8eef892774f87 for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,471 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/cfa6c2210933401bb7c7f9b187f92401 for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,477 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1f001470df6748aea71649cffaae8609 2023-08-07 08:53:44,479 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/690b4cd18b0d47bea9d964a7c786568b, top=false 2023-08-07 08:53:44,488 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e0d378647ec74ea1b52947731f7efefb 2023-08-07 08:53:44,493 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 48528e10cbed4040851175be536b961e 2023-08-07 08:53:44,496 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1f001470df6748aea71649cffaae8609 2023-08-07 08:53:44,496 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/1f001470df6748aea71649cffaae8609, top=true 2023-08-07 08:53:44,496 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/93712aa9572140b1b56e5833f35671e6, top=false 2023-08-07 08:53:44,497 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/75e432bba3d54ef486faa042619b92f8, top=false 2023-08-07 08:53:44,497 INFO [StoreFileSplitter-pool-7] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 876913088cb546d1b4701088081483d7 2023-08-07 08:53:44,502 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/39fc693feb564d94a22dd0a477a1d4f1, top=false 2023-08-07 08:53:44,502 DEBUG [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/a65272dd6d024b02b2d8eef892774f87, top=false 2023-08-07 08:53:44,503 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 233e2114e35449299d471e817e390162 2023-08-07 08:53:44,505 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cfa6c2210933401bb7c7f9b187f92401 2023-08-07 08:53:44,507 INFO [StoreFileSplitter-pool-11] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2f6d4ffab62d47d8b8a61ce088a54c99 2023-08-07 08:53:44,509 INFO [StoreFileSplitter-pool-7] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 876913088cb546d1b4701088081483d7 2023-08-07 08:53:44,509 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/876913088cb546d1b4701088081483d7, top=true 2023-08-07 08:53:44,510 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 48528e10cbed4040851175be536b961e 2023-08-07 08:53:44,510 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/48528e10cbed4040851175be536b961e, top=true 2023-08-07 08:53:44,511 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 233e2114e35449299d471e817e390162 2023-08-07 08:53:44,511 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/233e2114e35449299d471e817e390162, top=true 2023-08-07 08:53:44,513 INFO [StoreFileSplitter-pool-11] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2f6d4ffab62d47d8b8a61ce088a54c99 2023-08-07 08:53:44,513 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/2f6d4ffab62d47d8b8a61ce088a54c99, top=true 2023-08-07 08:53:44,539 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.splits/890310475de9dc263dd17491ca380a4c/cf2/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-690b4cd18b0d47bea9d964a7c786568b for child: 890310475de9dc263dd17491ca380a4c, parent: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,539 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.splits/890310475de9dc263dd17491ca380a4c/cf2/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-93712aa9572140b1b56e5833f35671e6 for child: 890310475de9dc263dd17491ca380a4c, parent: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,542 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/690b4cd18b0d47bea9d964a7c786568b for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,542 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/93712aa9572140b1b56e5833f35671e6 for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,542 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/d9bc9e401b7f42a2a6936beaf6b25a52 for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,542 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/ed4b0c71cd7d4265b3fdd040094d678d for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,545 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/d9bc9e401b7f42a2a6936beaf6b25a52, top=false 2023-08-07 08:53:44,550 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.splits/246cb6e862f1133bf06f7e32263c91b4/cf2/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-1f001470df6748aea71649cffaae8609 for child: 246cb6e862f1133bf06f7e32263c91b4, parent: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,551 INFO [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.splits/890310475de9dc263dd17491ca380a4c/cf1/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-a65272dd6d024b02b2d8eef892774f87 for child: 890310475de9dc263dd17491ca380a4c, parent: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,550 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.splits/890310475de9dc263dd17491ca380a4c/cf2/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-39fc693feb564d94a22dd0a477a1d4f1 for child: 890310475de9dc263dd17491ca380a4c, parent: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,551 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/1f001470df6748aea71649cffaae8609 for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,551 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/fb927cf3b10e4f6bab7afd25906a1091 for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,551 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.splits/246cb6e862f1133bf06f7e32263c91b4/cf2/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-48528e10cbed4040851175be536b961e for child: 246cb6e862f1133bf06f7e32263c91b4, parent: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,551 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.splits/246cb6e862f1133bf06f7e32263c91b4/cf1/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-2f6d4ffab62d47d8b8a61ce088a54c99 for child: 246cb6e862f1133bf06f7e32263c91b4, parent: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,551 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/48528e10cbed4040851175be536b961e for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,551 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/fe73c4030e5f45e5b2e8db40593817a7 for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,551 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/2f6d4ffab62d47d8b8a61ce088a54c99 for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,551 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.splits/246cb6e862f1133bf06f7e32263c91b4/cf1/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-233e2114e35449299d471e817e390162 for child: 246cb6e862f1133bf06f7e32263c91b4, parent: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,554 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/233e2114e35449299d471e817e390162 for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,555 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fb927cf3b10e4f6bab7afd25906a1091 2023-08-07 08:53:44,555 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/39fc693feb564d94a22dd0a477a1d4f1 for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,556 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/a65272dd6d024b02b2d8eef892774f87 for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,558 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fb927cf3b10e4f6bab7afd25906a1091 2023-08-07 08:53:44,558 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/fb927cf3b10e4f6bab7afd25906a1091, top=true 2023-08-07 08:53:44,562 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.splits/246cb6e862f1133bf06f7e32263c91b4/cf2/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-876913088cb546d1b4701088081483d7 for child: 246cb6e862f1133bf06f7e32263c91b4, parent: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,563 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/876913088cb546d1b4701088081483d7 for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,574 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.splits/890310475de9dc263dd17491ca380a4c/cf1/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-75e432bba3d54ef486faa042619b92f8 for child: 890310475de9dc263dd17491ca380a4c, parent: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,605 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/75e432bba3d54ef486faa042619b92f8 for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,615 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.splits/246cb6e862f1133bf06f7e32263c91b4/cf1/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-fb927cf3b10e4f6bab7afd25906a1091 for child: 246cb6e862f1133bf06f7e32263c91b4, parent: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,615 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/fb927cf3b10e4f6bab7afd25906a1091 for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,620 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e0d378647ec74ea1b52947731f7efefb 2023-08-07 08:53:44,627 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/.splits/890310475de9dc263dd17491ca380a4c/cf1/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-d9bc9e401b7f42a2a6936beaf6b25a52 for child: 890310475de9dc263dd17491ca380a4c, parent: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,650 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/d9bc9e401b7f42a2a6936beaf6b25a52 for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,651 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cfa6c2210933401bb7c7f9b187f92401 2023-08-07 08:53:44,664 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/68bc7551705944cb8c4eaeb76c784f25 for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,665 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/3a64bb0e4a9e41339304ef7ecb5b5d7a for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,667 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/4f5f12f2155941219ae22b6ac5110593 for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,672 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/7228be319c5144239680685192b85fb5 for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,672 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/e0d378647ec74ea1b52947731f7efefb for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,673 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/cfa6c2210933401bb7c7f9b187f92401 for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,675 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/ed4b0c71cd7d4265b3fdd040094d678d for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,678 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/fe73c4030e5f45e5b2e8db40593817a7 for region: 0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:44,678 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region 0d0fd872f4eb265c927f025c1f8dd4da Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-08-07 08:53:44,722 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-08-07 08:53:44,724 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-08-07 08:53:44,726 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1691398424725"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1691398424725"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1691398424725"}]},"ts":"1691398424725"} 2023-08-07 08:53:44,726 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1691398424725"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398424725"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398424725"}]},"ts":"1691398424725"} 2023-08-07 08:53:44,726 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1691398424725"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398424725"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398424725"}]},"ts":"1691398424725"} 2023-08-07 08:53:44,729 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=890310475de9dc263dd17491ca380a4c, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=246cb6e862f1133bf06f7e32263c91b4, ASSIGN}] 2023-08-07 08:53:44,730 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=246cb6e862f1133bf06f7e32263c91b4, ASSIGN 2023-08-07 08:53:44,730 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=246cb6e862f1133bf06f7e32263c91b4, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,34559,1691398396467; forceNewPlan=false, retain=false 2023-08-07 08:53:44,731 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=890310475de9dc263dd17491ca380a4c, ASSIGN 2023-08-07 08:53:44,732 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=890310475de9dc263dd17491ca380a4c, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,34559,1691398396467; forceNewPlan=false, retain=false 2023-08-07 08:53:44,881 INFO [jenkins-hbase4:33357] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-08-07 08:53:44,883 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=890310475de9dc263dd17491ca380a4c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:44,883 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1691398424883"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398424883"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398424883"}]},"ts":"1691398424883"} 2023-08-07 08:53:44,885 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=75, state=RUNNABLE; OpenRegionProcedure 890310475de9dc263dd17491ca380a4c, server=jenkins-hbase4.apache.org,34559,1691398396467}] 2023-08-07 08:53:44,887 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=246cb6e862f1133bf06f7e32263c91b4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:44,887 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1691398424887"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398424887"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398424887"}]},"ts":"1691398424887"} 2023-08-07 08:53:44,888 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=76, state=RUNNABLE; OpenRegionProcedure 246cb6e862f1133bf06f7e32263c91b4, server=jenkins-hbase4.apache.org,34559,1691398396467}] 2023-08-07 08:53:44,971 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-08-07 08:53:45,039 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c. 2023-08-07 08:53:45,039 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 890310475de9dc263dd17491ca380a4c, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c.', STARTKEY => '', ENDKEY => '60'} 2023-08-07 08:53:45,040 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-07 08:53:45,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 890310475de9dc263dd17491ca380a4c 2023-08-07 08:53:45,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:53:45,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 890310475de9dc263dd17491ca380a4c 2023-08-07 08:53:45,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 890310475de9dc263dd17491ca380a4c 2023-08-07 08:53:45,041 INFO [StoreOpener-890310475de9dc263dd17491ca380a4c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 890310475de9dc263dd17491ca380a4c 2023-08-07 08:53:45,042 DEBUG [StoreOpener-890310475de9dc263dd17491ca380a4c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1 2023-08-07 08:53:45,042 DEBUG [StoreOpener-890310475de9dc263dd17491ca380a4c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1 2023-08-07 08:53:45,042 INFO [StoreOpener-890310475de9dc263dd17491ca380a4c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 890310475de9dc263dd17491ca380a4c columnFamilyName cf1 2023-08-07 08:53:45,053 DEBUG [StoreOpener-890310475de9dc263dd17491ca380a4c-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/3a64bb0e4a9e41339304ef7ecb5b5d7a.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/3a64bb0e4a9e41339304ef7ecb5b5d7a-bottom 2023-08-07 08:53:45,056 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cfa6c2210933401bb7c7f9b187f92401.0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:45,057 DEBUG [StoreOpener-890310475de9dc263dd17491ca380a4c-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/cfa6c2210933401bb7c7f9b187f92401.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/cfa6c2210933401bb7c7f9b187f92401-bottom 2023-08-07 08:53:45,061 DEBUG [StoreOpener-890310475de9dc263dd17491ca380a4c-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/ed4b0c71cd7d4265b3fdd040094d678d.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/ed4b0c71cd7d4265b3fdd040094d678d-bottom 2023-08-07 08:53:45,065 DEBUG [StoreOpener-890310475de9dc263dd17491ca380a4c-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/fe73c4030e5f45e5b2e8db40593817a7.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/fe73c4030e5f45e5b2e8db40593817a7-bottom 2023-08-07 08:53:45,069 DEBUG [StoreOpener-890310475de9dc263dd17491ca380a4c-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-75e432bba3d54ef486faa042619b92f8 2023-08-07 08:53:45,073 DEBUG [StoreOpener-890310475de9dc263dd17491ca380a4c-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-a65272dd6d024b02b2d8eef892774f87 2023-08-07 08:53:45,077 DEBUG [StoreOpener-890310475de9dc263dd17491ca380a4c-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-d9bc9e401b7f42a2a6936beaf6b25a52 2023-08-07 08:53:45,077 INFO [StoreOpener-890310475de9dc263dd17491ca380a4c-1] regionserver.HStore(310): Store=890310475de9dc263dd17491ca380a4c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:45,077 INFO [StoreOpener-890310475de9dc263dd17491ca380a4c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 890310475de9dc263dd17491ca380a4c 2023-08-07 08:53:45,078 DEBUG [StoreOpener-890310475de9dc263dd17491ca380a4c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2 2023-08-07 08:53:45,078 DEBUG [StoreOpener-890310475de9dc263dd17491ca380a4c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2 2023-08-07 08:53:45,079 INFO [StoreOpener-890310475de9dc263dd17491ca380a4c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 890310475de9dc263dd17491ca380a4c columnFamilyName cf2 2023-08-07 08:53:45,092 DEBUG [StoreOpener-890310475de9dc263dd17491ca380a4c-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/4f5f12f2155941219ae22b6ac5110593.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/4f5f12f2155941219ae22b6ac5110593-bottom 2023-08-07 08:53:45,098 DEBUG [StoreOpener-890310475de9dc263dd17491ca380a4c-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/68bc7551705944cb8c4eaeb76c784f25.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/68bc7551705944cb8c4eaeb76c784f25-bottom 2023-08-07 08:53:45,102 DEBUG [StoreOpener-890310475de9dc263dd17491ca380a4c-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/7228be319c5144239680685192b85fb5.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/7228be319c5144239680685192b85fb5-bottom 2023-08-07 08:53:45,106 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e0d378647ec74ea1b52947731f7efefb.0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:45,107 DEBUG [StoreOpener-890310475de9dc263dd17491ca380a4c-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/e0d378647ec74ea1b52947731f7efefb.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/e0d378647ec74ea1b52947731f7efefb-bottom 2023-08-07 08:53:45,110 DEBUG [StoreOpener-890310475de9dc263dd17491ca380a4c-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-39fc693feb564d94a22dd0a477a1d4f1 2023-08-07 08:53:45,113 DEBUG [StoreOpener-890310475de9dc263dd17491ca380a4c-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-690b4cd18b0d47bea9d964a7c786568b 2023-08-07 08:53:45,117 DEBUG [StoreOpener-890310475de9dc263dd17491ca380a4c-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-93712aa9572140b1b56e5833f35671e6 2023-08-07 08:53:45,117 INFO [StoreOpener-890310475de9dc263dd17491ca380a4c-1] regionserver.HStore(310): Store=890310475de9dc263dd17491ca380a4c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:45,118 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c 2023-08-07 08:53:45,119 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c 2023-08-07 08:53:45,120 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-07 08:53:45,121 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 890310475de9dc263dd17491ca380a4c 2023-08-07 08:53:45,122 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 890310475de9dc263dd17491ca380a4c; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61344214, jitterRate=-0.08589997887611389}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-07 08:53:45,122 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 890310475de9dc263dd17491ca380a4c: 2023-08-07 08:53:45,123 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c., pid=77, masterSystemTime=1691398425036 2023-08-07 08:53:45,123 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:53:45,125 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-07 08:53:45,125 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-07 08:53:45,125 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-07 08:53:45,125 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c. 2023-08-07 08:53:45,125 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HStore(1912): 890310475de9dc263dd17491ca380a4c/cf1 is initiating minor compaction (all files) 2023-08-07 08:53:45,125 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c. 2023-08-07 08:53:45,125 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 890310475de9dc263dd17491ca380a4c/cf1 in testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c. 2023-08-07 08:53:45,125 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HStore(1912): 890310475de9dc263dd17491ca380a4c/cf2 is initiating minor compaction (all files) 2023-08-07 08:53:45,125 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 890310475de9dc263dd17491ca380a4c/cf2 in testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c. 2023-08-07 08:53:45,126 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/3a64bb0e4a9e41339304ef7ecb5b5d7a.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/3a64bb0e4a9e41339304ef7ecb5b5d7a-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/ed4b0c71cd7d4265b3fdd040094d678d.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/ed4b0c71cd7d4265b3fdd040094d678d-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/fe73c4030e5f45e5b2e8db40593817a7.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/fe73c4030e5f45e5b2e8db40593817a7-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-d9bc9e401b7f42a2a6936beaf6b25a52, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-a65272dd6d024b02b2d8eef892774f87, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-75e432bba3d54ef486faa042619b92f8, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/cfa6c2210933401bb7c7f9b187f92401.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/cfa6c2210933401bb7c7f9b187f92401-bottom] into tmpdir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/.tmp, totalSize=35.0 K 2023-08-07 08:53:45,126 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/68bc7551705944cb8c4eaeb76c784f25.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/68bc7551705944cb8c4eaeb76c784f25-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/4f5f12f2155941219ae22b6ac5110593.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/4f5f12f2155941219ae22b6ac5110593-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/7228be319c5144239680685192b85fb5.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/7228be319c5144239680685192b85fb5-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-39fc693feb564d94a22dd0a477a1d4f1, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-93712aa9572140b1b56e5833f35671e6, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-690b4cd18b0d47bea9d964a7c786568b, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/e0d378647ec74ea1b52947731f7efefb.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/e0d378647ec74ea1b52947731f7efefb-bottom] into tmpdir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/.tmp, totalSize=35.0 K 2023-08-07 08:53:45,127 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c. 2023-08-07 08:53:45,127 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting 3a64bb0e4a9e41339304ef7ecb5b5d7a.0d0fd872f4eb265c927f025c1f8dd4da, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1691398422572 2023-08-07 08:53:45,127 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c. 2023-08-07 08:53:45,127 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4. 2023-08-07 08:53:45,127 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting 68bc7551705944cb8c4eaeb76c784f25.0d0fd872f4eb265c927f025c1f8dd4da, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1691398422572 2023-08-07 08:53:45,127 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 246cb6e862f1133bf06f7e32263c91b4, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4.', STARTKEY => '60', ENDKEY => ''} 2023-08-07 08:53:45,127 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=890310475de9dc263dd17491ca380a4c, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:45,128 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1691398425127"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691398425127"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691398425127"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398425127"}]},"ts":"1691398425127"} 2023-08-07 08:53:45,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-07 08:53:45,128 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 246cb6e862f1133bf06f7e32263c91b4 2023-08-07 08:53:45,128 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:53:45,128 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 246cb6e862f1133bf06f7e32263c91b4 2023-08-07 08:53:45,128 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 246cb6e862f1133bf06f7e32263c91b4 2023-08-07 08:53:45,129 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting ed4b0c71cd7d4265b3fdd040094d678d.0d0fd872f4eb265c927f025c1f8dd4da, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1691398422766 2023-08-07 08:53:45,130 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting 4f5f12f2155941219ae22b6ac5110593.0d0fd872f4eb265c927f025c1f8dd4da, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1691398422766 2023-08-07 08:53:45,130 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting fe73c4030e5f45e5b2e8db40593817a7.0d0fd872f4eb265c927f025c1f8dd4da, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1691398422975 2023-08-07 08:53:45,130 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting 7228be319c5144239680685192b85fb5.0d0fd872f4eb265c927f025c1f8dd4da, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1691398422975 2023-08-07 08:53:45,131 INFO [StoreOpener-246cb6e862f1133bf06f7e32263c91b4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 246cb6e862f1133bf06f7e32263c91b4 2023-08-07 08:53:45,131 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=75 2023-08-07 08:53:45,131 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-d9bc9e401b7f42a2a6936beaf6b25a52, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1691398423155 2023-08-07 08:53:45,131 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=75, state=SUCCESS; OpenRegionProcedure 890310475de9dc263dd17491ca380a4c, server=jenkins-hbase4.apache.org,34559,1691398396467 in 244 msec 2023-08-07 08:53:45,131 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-39fc693feb564d94a22dd0a477a1d4f1, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1691398423155 2023-08-07 08:53:45,132 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-a65272dd6d024b02b2d8eef892774f87, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1691398423343 2023-08-07 08:53:45,132 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-93712aa9572140b1b56e5833f35671e6, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1691398423343 2023-08-07 08:53:45,132 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-75e432bba3d54ef486faa042619b92f8, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1691398423526 2023-08-07 08:53:45,132 DEBUG [StoreOpener-246cb6e862f1133bf06f7e32263c91b4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1 2023-08-07 08:53:45,132 DEBUG [StoreOpener-246cb6e862f1133bf06f7e32263c91b4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1 2023-08-07 08:53:45,133 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-690b4cd18b0d47bea9d964a7c786568b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1691398423526 2023-08-07 08:53:45,133 INFO [StoreOpener-246cb6e862f1133bf06f7e32263c91b4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 246cb6e862f1133bf06f7e32263c91b4 columnFamilyName cf1 2023-08-07 08:53:45,133 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting cfa6c2210933401bb7c7f9b187f92401.0d0fd872f4eb265c927f025c1f8dd4da, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1691398423708 2023-08-07 08:53:45,133 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=890310475de9dc263dd17491ca380a4c, ASSIGN in 402 msec 2023-08-07 08:53:45,133 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting e0d378647ec74ea1b52947731f7efefb.0d0fd872f4eb265c927f025c1f8dd4da, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1691398423708 2023-08-07 08:53:45,143 DEBUG [StoreOpener-246cb6e862f1133bf06f7e32263c91b4-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/3a64bb0e4a9e41339304ef7ecb5b5d7a.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/3a64bb0e4a9e41339304ef7ecb5b5d7a-top 2023-08-07 08:53:45,147 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cfa6c2210933401bb7c7f9b187f92401.0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:45,147 DEBUG [StoreOpener-246cb6e862f1133bf06f7e32263c91b4-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/cfa6c2210933401bb7c7f9b187f92401.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/cfa6c2210933401bb7c7f9b187f92401-top 2023-08-07 08:53:45,148 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] throttle.PressureAwareThroughputController(145): 890310475de9dc263dd17491ca380a4c#cf1#compaction#72 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-07 08:53:45,151 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] throttle.PressureAwareThroughputController(145): 890310475de9dc263dd17491ca380a4c#cf2#compaction#73 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-07 08:53:45,154 DEBUG [StoreOpener-246cb6e862f1133bf06f7e32263c91b4-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/ed4b0c71cd7d4265b3fdd040094d678d.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/ed4b0c71cd7d4265b3fdd040094d678d-top 2023-08-07 08:53:45,163 DEBUG [StoreOpener-246cb6e862f1133bf06f7e32263c91b4-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/fe73c4030e5f45e5b2e8db40593817a7.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/fe73c4030e5f45e5b2e8db40593817a7-top 2023-08-07 08:53:45,173 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 233e2114e35449299d471e817e390162 2023-08-07 08:53:45,173 DEBUG [StoreOpener-246cb6e862f1133bf06f7e32263c91b4-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-233e2114e35449299d471e817e390162 2023-08-07 08:53:45,176 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/.tmp/cf2/df02dd8a38a648aa9b81cd6caa710f25 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/df02dd8a38a648aa9b81cd6caa710f25 2023-08-07 08:53:45,177 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2f6d4ffab62d47d8b8a61ce088a54c99 2023-08-07 08:53:45,177 DEBUG [StoreOpener-246cb6e862f1133bf06f7e32263c91b4-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-2f6d4ffab62d47d8b8a61ce088a54c99 2023-08-07 08:53:45,179 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/.tmp/cf1/eebff97650504cb78793d709d2bd9936 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/eebff97650504cb78793d709d2bd9936 2023-08-07 08:53:45,182 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fb927cf3b10e4f6bab7afd25906a1091 2023-08-07 08:53:45,182 DEBUG [StoreOpener-246cb6e862f1133bf06f7e32263c91b4-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-fb927cf3b10e4f6bab7afd25906a1091 2023-08-07 08:53:45,182 INFO [StoreOpener-246cb6e862f1133bf06f7e32263c91b4-1] regionserver.HStore(310): Store=246cb6e862f1133bf06f7e32263c91b4/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:45,182 INFO [StoreOpener-246cb6e862f1133bf06f7e32263c91b4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 246cb6e862f1133bf06f7e32263c91b4 2023-08-07 08:53:45,184 DEBUG [StoreOpener-246cb6e862f1133bf06f7e32263c91b4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2 2023-08-07 08:53:45,184 DEBUG [StoreOpener-246cb6e862f1133bf06f7e32263c91b4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2 2023-08-07 08:53:45,184 INFO [StoreOpener-246cb6e862f1133bf06f7e32263c91b4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 246cb6e862f1133bf06f7e32263c91b4 columnFamilyName cf2 2023-08-07 08:53:45,185 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 890310475de9dc263dd17491ca380a4c/cf2 of 890310475de9dc263dd17491ca380a4c into df02dd8a38a648aa9b81cd6caa710f25(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 08:53:45,185 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 890310475de9dc263dd17491ca380a4c: 2023-08-07 08:53:45,185 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c., storeName=890310475de9dc263dd17491ca380a4c/cf2, priority=9, startTime=1691398425125; duration=0sec 2023-08-07 08:53:45,185 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:53:45,185 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 890310475de9dc263dd17491ca380a4c/cf1 of 890310475de9dc263dd17491ca380a4c into eebff97650504cb78793d709d2bd9936(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 08:53:45,185 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 890310475de9dc263dd17491ca380a4c: 2023-08-07 08:53:45,185 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c., storeName=890310475de9dc263dd17491ca380a4c/cf1, priority=9, startTime=1691398425123; duration=0sec 2023-08-07 08:53:45,186 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:53:45,194 DEBUG [StoreOpener-246cb6e862f1133bf06f7e32263c91b4-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/4f5f12f2155941219ae22b6ac5110593.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/4f5f12f2155941219ae22b6ac5110593-top 2023-08-07 08:53:45,196 DEBUG [StoreOpener-246cb6e862f1133bf06f7e32263c91b4-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/68bc7551705944cb8c4eaeb76c784f25.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/68bc7551705944cb8c4eaeb76c784f25-top 2023-08-07 08:53:45,199 DEBUG [StoreOpener-246cb6e862f1133bf06f7e32263c91b4-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/7228be319c5144239680685192b85fb5.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/7228be319c5144239680685192b85fb5-top 2023-08-07 08:53:45,202 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e0d378647ec74ea1b52947731f7efefb.0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:45,202 DEBUG [StoreOpener-246cb6e862f1133bf06f7e32263c91b4-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/e0d378647ec74ea1b52947731f7efefb.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/e0d378647ec74ea1b52947731f7efefb-top 2023-08-07 08:53:45,205 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1f001470df6748aea71649cffaae8609 2023-08-07 08:53:45,205 DEBUG [StoreOpener-246cb6e862f1133bf06f7e32263c91b4-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-1f001470df6748aea71649cffaae8609 2023-08-07 08:53:45,207 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 48528e10cbed4040851175be536b961e 2023-08-07 08:53:45,207 DEBUG [StoreOpener-246cb6e862f1133bf06f7e32263c91b4-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-48528e10cbed4040851175be536b961e 2023-08-07 08:53:45,210 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 876913088cb546d1b4701088081483d7 2023-08-07 08:53:45,210 DEBUG [StoreOpener-246cb6e862f1133bf06f7e32263c91b4-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-876913088cb546d1b4701088081483d7 2023-08-07 08:53:45,210 INFO [StoreOpener-246cb6e862f1133bf06f7e32263c91b4-1] regionserver.HStore(310): Store=246cb6e862f1133bf06f7e32263c91b4/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:45,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4 2023-08-07 08:53:45,212 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4 2023-08-07 08:53:45,213 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-07 08:53:45,214 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 246cb6e862f1133bf06f7e32263c91b4 2023-08-07 08:53:45,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 246cb6e862f1133bf06f7e32263c91b4; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61638158, jitterRate=-0.08151987195014954}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-07 08:53:45,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 246cb6e862f1133bf06f7e32263c91b4: 2023-08-07 08:53:45,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4., pid=78, masterSystemTime=1691398425036 2023-08-07 08:53:45,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:53:45,217 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-07 08:53:45,217 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-07 08:53:45,217 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-07 08:53:45,217 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4. 2023-08-07 08:53:45,217 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4. 2023-08-07 08:53:45,217 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HStore(1912): 246cb6e862f1133bf06f7e32263c91b4/cf1 is initiating minor compaction (all files) 2023-08-07 08:53:45,217 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HStore(1912): 246cb6e862f1133bf06f7e32263c91b4/cf2 is initiating minor compaction (all files) 2023-08-07 08:53:45,217 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 246cb6e862f1133bf06f7e32263c91b4/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4. 2023-08-07 08:53:45,217 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 246cb6e862f1133bf06f7e32263c91b4/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4. 2023-08-07 08:53:45,217 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/68bc7551705944cb8c4eaeb76c784f25.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/68bc7551705944cb8c4eaeb76c784f25-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/4f5f12f2155941219ae22b6ac5110593.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/4f5f12f2155941219ae22b6ac5110593-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/7228be319c5144239680685192b85fb5.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/7228be319c5144239680685192b85fb5-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/e0d378647ec74ea1b52947731f7efefb.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/e0d378647ec74ea1b52947731f7efefb-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-876913088cb546d1b4701088081483d7, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-48528e10cbed4040851175be536b961e, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-1f001470df6748aea71649cffaae8609] into tmpdir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/.tmp, totalSize=34.8 K 2023-08-07 08:53:45,217 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/3a64bb0e4a9e41339304ef7ecb5b5d7a.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/3a64bb0e4a9e41339304ef7ecb5b5d7a-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/ed4b0c71cd7d4265b3fdd040094d678d.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/ed4b0c71cd7d4265b3fdd040094d678d-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/fe73c4030e5f45e5b2e8db40593817a7.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/fe73c4030e5f45e5b2e8db40593817a7-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/cfa6c2210933401bb7c7f9b187f92401.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/cfa6c2210933401bb7c7f9b187f92401-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-fb927cf3b10e4f6bab7afd25906a1091, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-2f6d4ffab62d47d8b8a61ce088a54c99, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-233e2114e35449299d471e817e390162] into tmpdir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/.tmp, totalSize=34.8 K 2023-08-07 08:53:45,218 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting 68bc7551705944cb8c4eaeb76c784f25.0d0fd872f4eb265c927f025c1f8dd4da, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1691398422572 2023-08-07 08:53:45,218 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting 3a64bb0e4a9e41339304ef7ecb5b5d7a.0d0fd872f4eb265c927f025c1f8dd4da, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1691398422572 2023-08-07 08:53:45,218 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4. 2023-08-07 08:53:45,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4. 2023-08-07 08:53:45,219 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting 4f5f12f2155941219ae22b6ac5110593.0d0fd872f4eb265c927f025c1f8dd4da, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1691398422766 2023-08-07 08:53:45,219 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting ed4b0c71cd7d4265b3fdd040094d678d.0d0fd872f4eb265c927f025c1f8dd4da, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1691398422766 2023-08-07 08:53:45,219 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=246cb6e862f1133bf06f7e32263c91b4, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:45,219 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1691398425219"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691398425219"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691398425219"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398425219"}]},"ts":"1691398425219"} 2023-08-07 08:53:45,220 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting 7228be319c5144239680685192b85fb5.0d0fd872f4eb265c927f025c1f8dd4da, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1691398422975 2023-08-07 08:53:45,220 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting fe73c4030e5f45e5b2e8db40593817a7.0d0fd872f4eb265c927f025c1f8dd4da, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1691398422975 2023-08-07 08:53:45,220 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting e0d378647ec74ea1b52947731f7efefb.0d0fd872f4eb265c927f025c1f8dd4da, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1691398423708 2023-08-07 08:53:45,220 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting cfa6c2210933401bb7c7f9b187f92401.0d0fd872f4eb265c927f025c1f8dd4da, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1691398423708 2023-08-07 08:53:45,220 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-876913088cb546d1b4701088081483d7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-08-07 08:53:45,221 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-fb927cf3b10e4f6bab7afd25906a1091, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-08-07 08:53:45,221 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-48528e10cbed4040851175be536b961e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-08-07 08:53:45,221 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-2f6d4ffab62d47d8b8a61ce088a54c99, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-08-07 08:53:45,221 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-233e2114e35449299d471e817e390162, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-08-07 08:53:45,222 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-1f001470df6748aea71649cffaae8609, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-08-07 08:53:45,223 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=76 2023-08-07 08:53:45,223 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=76, state=SUCCESS; OpenRegionProcedure 246cb6e862f1133bf06f7e32263c91b4, server=jenkins-hbase4.apache.org,34559,1691398396467 in 333 msec 2023-08-07 08:53:45,225 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=72 2023-08-07 08:53:45,225 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=246cb6e862f1133bf06f7e32263c91b4, ASSIGN in 494 msec 2023-08-07 08:53:45,227 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0d0fd872f4eb265c927f025c1f8dd4da, daughterA=890310475de9dc263dd17491ca380a4c, daughterB=246cb6e862f1133bf06f7e32263c91b4 in 991 msec 2023-08-07 08:53:45,238 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] throttle.PressureAwareThroughputController(145): 246cb6e862f1133bf06f7e32263c91b4#cf1#compaction#74 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-07 08:53:45,238 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] throttle.PressureAwareThroughputController(145): 246cb6e862f1133bf06f7e32263c91b4#cf2#compaction#75 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-07 08:53:45,256 DEBUG [Listener at localhost/33561] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-07 08:53:45,264 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42940, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-07 08:53:45,264 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34559] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c. 2023-08-07 08:53:45,265 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34559] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-08-07 08:53:45,265 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34559] regionserver.HStore(1912): 890310475de9dc263dd17491ca380a4c/cf1 is initiating major compaction (all files) 2023-08-07 08:53:45,265 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34559] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@2d7f55a2; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-07 08:53:45,265 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34559] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-08-07 08:53:45,265 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34559] regionserver.HStore(1912): 890310475de9dc263dd17491ca380a4c/cf2 is initiating major compaction (all files) 2023-08-07 08:53:45,265 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34559] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@47bd6ced; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c.; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-08-07 08:53:45,266 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 0365a3e4675143368e07f01620bbfdde: NONE, but ROW specified in column family configuration 2023-08-07 08:53:45,267 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/.tmp/cf1/0365a3e4675143368e07f01620bbfdde as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/0365a3e4675143368e07f01620bbfdde 2023-08-07 08:53:45,267 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34559] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4. 2023-08-07 08:53:45,267 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34559] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 7 compacting, 0 eligible, 16 blocking 2023-08-07 08:53:45,268 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34559] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-08-07 08:53:45,268 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34559] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-08-07 08:53:45,268 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34559] regionserver.CompactSplit(415): Not compacting testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4. because compaction request was cancelled 2023-08-07 08:53:45,268 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34559] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 7 compacting, 0 eligible, 16 blocking 2023-08-07 08:53:45,268 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34559] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-08-07 08:53:45,268 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34559] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-08-07 08:53:45,268 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34559] regionserver.CompactSplit(415): Not compacting testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4. because compaction request was cancelled 2023-08-07 08:53:45,269 INFO [Listener at localhost/33561] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-08-07 08:53:45,271 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 47af2313c26d46c895bcd9c7b4ea29f4: NONE, but ROW specified in column family configuration 2023-08-07 08:53:45,272 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/.tmp/cf2/47af2313c26d46c895bcd9c7b4ea29f4 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/47af2313c26d46c895bcd9c7b4ea29f4 2023-08-07 08:53:45,272 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 0365a3e4675143368e07f01620bbfdde: NONE, but ROW specified in column family configuration 2023-08-07 08:53:45,273 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 246cb6e862f1133bf06f7e32263c91b4/cf1 of 246cb6e862f1133bf06f7e32263c91b4 into 0365a3e4675143368e07f01620bbfdde(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 08:53:45,273 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 246cb6e862f1133bf06f7e32263c91b4: 2023-08-07 08:53:45,273 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4., storeName=246cb6e862f1133bf06f7e32263c91b4/cf1, priority=9, startTime=1691398425215; duration=0sec 2023-08-07 08:53:45,274 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-08-07 08:53:45,274 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 890310475de9dc263dd17491ca380a4c/cf2 in testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c. 2023-08-07 08:53:45,274 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/df02dd8a38a648aa9b81cd6caa710f25] into tmpdir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/.tmp, totalSize=6.8 K 2023-08-07 08:53:45,274 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting df02dd8a38a648aa9b81cd6caa710f25, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1691398422572 2023-08-07 08:53:45,277 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 47af2313c26d46c895bcd9c7b4ea29f4: NONE, but ROW specified in column family configuration 2023-08-07 08:53:45,278 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 246cb6e862f1133bf06f7e32263c91b4/cf2 of 246cb6e862f1133bf06f7e32263c91b4 into 47af2313c26d46c895bcd9c7b4ea29f4(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 08:53:45,278 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 246cb6e862f1133bf06f7e32263c91b4: 2023-08-07 08:53:45,278 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4., storeName=246cb6e862f1133bf06f7e32263c91b4/cf2, priority=9, startTime=1691398425217; duration=0sec 2023-08-07 08:53:45,278 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-07 08:53:45,278 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 890310475de9dc263dd17491ca380a4c/cf1 in testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c. 2023-08-07 08:53:45,278 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/eebff97650504cb78793d709d2bd9936] into tmpdir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/.tmp, totalSize=6.8 K 2023-08-07 08:53:45,279 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting eebff97650504cb78793d709d2bd9936, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1691398422572 2023-08-07 08:53:45,283 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] throttle.PressureAwareThroughputController(145): 890310475de9dc263dd17491ca380a4c#cf1#compaction#76 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-07 08:53:45,291 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] throttle.PressureAwareThroughputController(145): 890310475de9dc263dd17491ca380a4c#cf2#compaction#77 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-07 08:53:45,293 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34559] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c. is not mergeable because it has references 2023-08-07 08:53:45,294 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34559] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4. is not mergeable because it has references 2023-08-07 08:53:45,308 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/.tmp/cf1/d7b7659d2ee2484b8b7cb16f325c20f3 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/d7b7659d2ee2484b8b7cb16f325c20f3 2023-08-07 08:53:45,310 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/.tmp/cf2/c1a81b9ee15f4d3593681e2c8e47e577 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/c1a81b9ee15f4d3593681e2c8e47e577 2023-08-07 08:53:45,313 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 890310475de9dc263dd17491ca380a4c/cf1 of 890310475de9dc263dd17491ca380a4c into d7b7659d2ee2484b8b7cb16f325c20f3(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 08:53:45,313 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 890310475de9dc263dd17491ca380a4c: 2023-08-07 08:53:45,314 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c., storeName=cf1, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1691398425265; duration=0sec 2023-08-07 08:53:45,314 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:53:45,315 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 890310475de9dc263dd17491ca380a4c/cf2 of 890310475de9dc263dd17491ca380a4c into c1a81b9ee15f4d3593681e2c8e47e577(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 08:53:45,315 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 890310475de9dc263dd17491ca380a4c: 2023-08-07 08:53:45,315 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c., storeName=cf2, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1691398425265; duration=0sec 2023-08-07 08:53:45,316 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:53:45,402 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34559] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c. is not mergeable because it has references 2023-08-07 08:53:45,403 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34559] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4. is not mergeable because it has references 2023-08-07 08:53:45,409 DEBUG [Listener at localhost/33561] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-08-07 08:53:45,410 WARN [Listener at localhost/33561] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-07 08:53:45,412 INFO [Listener at localhost/33561] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:45,412 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:45,413 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:45,415 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-08-07 08:53:45,415 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398425415"}]},"ts":"1691398425415"} 2023-08-07 08:53:45,416 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-08-07 08:53:45,418 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-08-07 08:53:45,419 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=890310475de9dc263dd17491ca380a4c, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=246cb6e862f1133bf06f7e32263c91b4, UNASSIGN}] 2023-08-07 08:53:45,419 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=246cb6e862f1133bf06f7e32263c91b4, UNASSIGN 2023-08-07 08:53:45,420 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=246cb6e862f1133bf06f7e32263c91b4, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:45,420 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1691398425420"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398425420"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398425420"}]},"ts":"1691398425420"} 2023-08-07 08:53:45,421 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure 246cb6e862f1133bf06f7e32263c91b4, server=jenkins-hbase4.apache.org,34559,1691398396467}] 2023-08-07 08:53:45,423 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=890310475de9dc263dd17491ca380a4c, UNASSIGN 2023-08-07 08:53:45,424 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=890310475de9dc263dd17491ca380a4c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:45,424 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1691398425424"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398425424"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398425424"}]},"ts":"1691398425424"} 2023-08-07 08:53:45,425 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure 890310475de9dc263dd17491ca380a4c, server=jenkins-hbase4.apache.org,34559,1691398396467}] 2023-08-07 08:53:45,516 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-08-07 08:53:45,573 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 246cb6e862f1133bf06f7e32263c91b4 2023-08-07 08:53:45,573 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 246cb6e862f1133bf06f7e32263c91b4, disabling compactions & flushes 2023-08-07 08:53:45,573 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4. 2023-08-07 08:53:45,573 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4. 2023-08-07 08:53:45,573 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4. after waiting 0 ms 2023-08-07 08:53:45,573 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4. 2023-08-07 08:53:45,574 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/3a64bb0e4a9e41339304ef7ecb5b5d7a.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/3a64bb0e4a9e41339304ef7ecb5b5d7a-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/ed4b0c71cd7d4265b3fdd040094d678d.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/ed4b0c71cd7d4265b3fdd040094d678d-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/fe73c4030e5f45e5b2e8db40593817a7.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/fe73c4030e5f45e5b2e8db40593817a7-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/cfa6c2210933401bb7c7f9b187f92401.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/cfa6c2210933401bb7c7f9b187f92401-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-fb927cf3b10e4f6bab7afd25906a1091, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-2f6d4ffab62d47d8b8a61ce088a54c99, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-233e2114e35449299d471e817e390162] to archive 2023-08-07 08:53:45,575 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-07 08:53:45,576 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/3a64bb0e4a9e41339304ef7ecb5b5d7a.0d0fd872f4eb265c927f025c1f8dd4da to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/3a64bb0e4a9e41339304ef7ecb5b5d7a.0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:45,577 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/ed4b0c71cd7d4265b3fdd040094d678d.0d0fd872f4eb265c927f025c1f8dd4da to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/ed4b0c71cd7d4265b3fdd040094d678d.0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:45,579 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/fe73c4030e5f45e5b2e8db40593817a7.0d0fd872f4eb265c927f025c1f8dd4da to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/fe73c4030e5f45e5b2e8db40593817a7.0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:45,580 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/cfa6c2210933401bb7c7f9b187f92401.0d0fd872f4eb265c927f025c1f8dd4da to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/cfa6c2210933401bb7c7f9b187f92401.0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:45,581 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-fb927cf3b10e4f6bab7afd25906a1091 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-fb927cf3b10e4f6bab7afd25906a1091 2023-08-07 08:53:45,582 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-2f6d4ffab62d47d8b8a61ce088a54c99 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-2f6d4ffab62d47d8b8a61ce088a54c99 2023-08-07 08:53:45,583 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-233e2114e35449299d471e817e390162 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-233e2114e35449299d471e817e390162 2023-08-07 08:53:45,585 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/68bc7551705944cb8c4eaeb76c784f25.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/68bc7551705944cb8c4eaeb76c784f25-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/4f5f12f2155941219ae22b6ac5110593.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/4f5f12f2155941219ae22b6ac5110593-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/7228be319c5144239680685192b85fb5.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/7228be319c5144239680685192b85fb5-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/e0d378647ec74ea1b52947731f7efefb.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/e0d378647ec74ea1b52947731f7efefb-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-876913088cb546d1b4701088081483d7, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-48528e10cbed4040851175be536b961e, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-1f001470df6748aea71649cffaae8609] to archive 2023-08-07 08:53:45,585 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-07 08:53:45,586 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/68bc7551705944cb8c4eaeb76c784f25.0d0fd872f4eb265c927f025c1f8dd4da to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/68bc7551705944cb8c4eaeb76c784f25.0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:45,587 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/4f5f12f2155941219ae22b6ac5110593.0d0fd872f4eb265c927f025c1f8dd4da to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/4f5f12f2155941219ae22b6ac5110593.0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:45,588 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/7228be319c5144239680685192b85fb5.0d0fd872f4eb265c927f025c1f8dd4da to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/7228be319c5144239680685192b85fb5.0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:45,589 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/e0d378647ec74ea1b52947731f7efefb.0d0fd872f4eb265c927f025c1f8dd4da to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/e0d378647ec74ea1b52947731f7efefb.0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:45,590 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-876913088cb546d1b4701088081483d7 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-876913088cb546d1b4701088081483d7 2023-08-07 08:53:45,591 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-48528e10cbed4040851175be536b961e to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-48528e10cbed4040851175be536b961e 2023-08-07 08:53:45,592 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-1f001470df6748aea71649cffaae8609 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-1f001470df6748aea71649cffaae8609 2023-08-07 08:53:45,596 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/recovered.edits/111.seqid, newMaxSeqId=111, maxSeqId=106 2023-08-07 08:53:45,596 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-07 08:53:45,596 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4. 2023-08-07 08:53:45,596 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 246cb6e862f1133bf06f7e32263c91b4: 2023-08-07 08:53:45,597 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 246cb6e862f1133bf06f7e32263c91b4 2023-08-07 08:53:45,598 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 890310475de9dc263dd17491ca380a4c 2023-08-07 08:53:45,598 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 890310475de9dc263dd17491ca380a4c, disabling compactions & flushes 2023-08-07 08:53:45,598 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c. 2023-08-07 08:53:45,598 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c. 2023-08-07 08:53:45,598 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c. after waiting 0 ms 2023-08-07 08:53:45,598 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=246cb6e862f1133bf06f7e32263c91b4, regionState=CLOSED 2023-08-07 08:53:45,598 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c. 2023-08-07 08:53:45,598 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1691398425598"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398425598"}]},"ts":"1691398425598"} 2023-08-07 08:53:45,599 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/3a64bb0e4a9e41339304ef7ecb5b5d7a.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/3a64bb0e4a9e41339304ef7ecb5b5d7a-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/ed4b0c71cd7d4265b3fdd040094d678d.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/ed4b0c71cd7d4265b3fdd040094d678d-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/fe73c4030e5f45e5b2e8db40593817a7.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/fe73c4030e5f45e5b2e8db40593817a7-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-d9bc9e401b7f42a2a6936beaf6b25a52, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-a65272dd6d024b02b2d8eef892774f87, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-75e432bba3d54ef486faa042619b92f8, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/eebff97650504cb78793d709d2bd9936, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/cfa6c2210933401bb7c7f9b187f92401.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/cfa6c2210933401bb7c7f9b187f92401-bottom] to archive 2023-08-07 08:53:45,600 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-07 08:53:45,601 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-08-07 08:53:45,601 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure 246cb6e862f1133bf06f7e32263c91b4, server=jenkins-hbase4.apache.org,34559,1691398396467 in 179 msec 2023-08-07 08:53:45,601 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/3a64bb0e4a9e41339304ef7ecb5b5d7a.0d0fd872f4eb265c927f025c1f8dd4da to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/3a64bb0e4a9e41339304ef7ecb5b5d7a.0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:45,602 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=246cb6e862f1133bf06f7e32263c91b4, UNASSIGN in 182 msec 2023-08-07 08:53:45,603 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/ed4b0c71cd7d4265b3fdd040094d678d.0d0fd872f4eb265c927f025c1f8dd4da to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/ed4b0c71cd7d4265b3fdd040094d678d.0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:45,604 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/fe73c4030e5f45e5b2e8db40593817a7.0d0fd872f4eb265c927f025c1f8dd4da to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/fe73c4030e5f45e5b2e8db40593817a7.0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:45,605 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-d9bc9e401b7f42a2a6936beaf6b25a52 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-d9bc9e401b7f42a2a6936beaf6b25a52 2023-08-07 08:53:45,606 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-a65272dd6d024b02b2d8eef892774f87 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-a65272dd6d024b02b2d8eef892774f87 2023-08-07 08:53:45,607 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-75e432bba3d54ef486faa042619b92f8 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-75e432bba3d54ef486faa042619b92f8 2023-08-07 08:53:45,608 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/eebff97650504cb78793d709d2bd9936 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/eebff97650504cb78793d709d2bd9936 2023-08-07 08:53:45,608 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/cfa6c2210933401bb7c7f9b187f92401.0d0fd872f4eb265c927f025c1f8dd4da to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/cfa6c2210933401bb7c7f9b187f92401.0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:45,610 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/68bc7551705944cb8c4eaeb76c784f25.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/68bc7551705944cb8c4eaeb76c784f25-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/4f5f12f2155941219ae22b6ac5110593.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/4f5f12f2155941219ae22b6ac5110593-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/7228be319c5144239680685192b85fb5.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/7228be319c5144239680685192b85fb5-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-39fc693feb564d94a22dd0a477a1d4f1, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-93712aa9572140b1b56e5833f35671e6, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-690b4cd18b0d47bea9d964a7c786568b, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/df02dd8a38a648aa9b81cd6caa710f25, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/e0d378647ec74ea1b52947731f7efefb.0d0fd872f4eb265c927f025c1f8dd4da->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/e0d378647ec74ea1b52947731f7efefb-bottom] to archive 2023-08-07 08:53:45,611 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-07 08:53:45,612 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/68bc7551705944cb8c4eaeb76c784f25.0d0fd872f4eb265c927f025c1f8dd4da to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/68bc7551705944cb8c4eaeb76c784f25.0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:45,613 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/4f5f12f2155941219ae22b6ac5110593.0d0fd872f4eb265c927f025c1f8dd4da to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/4f5f12f2155941219ae22b6ac5110593.0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:45,614 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/7228be319c5144239680685192b85fb5.0d0fd872f4eb265c927f025c1f8dd4da to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/7228be319c5144239680685192b85fb5.0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:45,615 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-39fc693feb564d94a22dd0a477a1d4f1 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-39fc693feb564d94a22dd0a477a1d4f1 2023-08-07 08:53:45,616 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-93712aa9572140b1b56e5833f35671e6 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-93712aa9572140b1b56e5833f35671e6 2023-08-07 08:53:45,617 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-690b4cd18b0d47bea9d964a7c786568b to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/testSplitTableRegionDeletedRowsDaughter=0d0fd872f4eb265c927f025c1f8dd4da-690b4cd18b0d47bea9d964a7c786568b 2023-08-07 08:53:45,618 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/df02dd8a38a648aa9b81cd6caa710f25 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/df02dd8a38a648aa9b81cd6caa710f25 2023-08-07 08:53:45,619 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/e0d378647ec74ea1b52947731f7efefb.0d0fd872f4eb265c927f025c1f8dd4da to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/e0d378647ec74ea1b52947731f7efefb.0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:45,622 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-08-07 08:53:45,622 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-07 08:53:45,623 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c. 2023-08-07 08:53:45,623 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 890310475de9dc263dd17491ca380a4c: 2023-08-07 08:53:45,624 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 890310475de9dc263dd17491ca380a4c 2023-08-07 08:53:45,624 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=890310475de9dc263dd17491ca380a4c, regionState=CLOSED 2023-08-07 08:53:45,624 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1691398425624"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398425624"}]},"ts":"1691398425624"} 2023-08-07 08:53:45,627 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-08-07 08:53:45,627 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure 890310475de9dc263dd17491ca380a4c, server=jenkins-hbase4.apache.org,34559,1691398396467 in 200 msec 2023-08-07 08:53:45,628 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=80, resume processing ppid=79 2023-08-07 08:53:45,628 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=890310475de9dc263dd17491ca380a4c, UNASSIGN in 208 msec 2023-08-07 08:53:45,629 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398425629"}]},"ts":"1691398425629"} 2023-08-07 08:53:45,630 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-08-07 08:53:45,631 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-08-07 08:53:45,633 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 220 msec 2023-08-07 08:53:45,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-08-07 08:53:45,718 INFO [Listener at localhost/33561] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-08-07 08:53:45,718 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:45,719 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:45,721 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:45,721 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=84 2023-08-07 08:53:45,721 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:45,726 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:45,726 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4 2023-08-07 08:53:45,726 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c 2023-08-07 08:53:45,728 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/recovered.edits] 2023-08-07 08:53:45,728 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/recovered.edits] 2023-08-07 08:53:45,728 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/recovered.edits] 2023-08-07 08:53:45,732 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/d7b7659d2ee2484b8b7cb16f325c20f3 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf1/d7b7659d2ee2484b8b7cb16f325c20f3 2023-08-07 08:53:45,732 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/0365a3e4675143368e07f01620bbfdde to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf1/0365a3e4675143368e07f01620bbfdde 2023-08-07 08:53:45,732 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/233e2114e35449299d471e817e390162 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/233e2114e35449299d471e817e390162 2023-08-07 08:53:45,734 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/2f6d4ffab62d47d8b8a61ce088a54c99 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/2f6d4ffab62d47d8b8a61ce088a54c99 2023-08-07 08:53:45,735 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/c1a81b9ee15f4d3593681e2c8e47e577 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/cf2/c1a81b9ee15f4d3593681e2c8e47e577 2023-08-07 08:53:45,735 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/47af2313c26d46c895bcd9c7b4ea29f4 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/cf2/47af2313c26d46c895bcd9c7b4ea29f4 2023-08-07 08:53:45,736 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/3a64bb0e4a9e41339304ef7ecb5b5d7a to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/3a64bb0e4a9e41339304ef7ecb5b5d7a 2023-08-07 08:53:45,737 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/75e432bba3d54ef486faa042619b92f8 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/75e432bba3d54ef486faa042619b92f8 2023-08-07 08:53:45,738 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/recovered.edits/113.seqid to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c/recovered.edits/113.seqid 2023-08-07 08:53:45,738 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/recovered.edits/111.seqid to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4/recovered.edits/111.seqid 2023-08-07 08:53:45,739 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/a65272dd6d024b02b2d8eef892774f87 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/a65272dd6d024b02b2d8eef892774f87 2023-08-07 08:53:45,739 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/890310475de9dc263dd17491ca380a4c 2023-08-07 08:53:45,739 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/246cb6e862f1133bf06f7e32263c91b4 2023-08-07 08:53:45,740 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/cfa6c2210933401bb7c7f9b187f92401 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/cfa6c2210933401bb7c7f9b187f92401 2023-08-07 08:53:45,742 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/d9bc9e401b7f42a2a6936beaf6b25a52 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/d9bc9e401b7f42a2a6936beaf6b25a52 2023-08-07 08:53:45,743 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/ed4b0c71cd7d4265b3fdd040094d678d to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/ed4b0c71cd7d4265b3fdd040094d678d 2023-08-07 08:53:45,744 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/fb927cf3b10e4f6bab7afd25906a1091 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/fb927cf3b10e4f6bab7afd25906a1091 2023-08-07 08:53:45,745 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/fe73c4030e5f45e5b2e8db40593817a7 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf1/fe73c4030e5f45e5b2e8db40593817a7 2023-08-07 08:53:45,747 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/1f001470df6748aea71649cffaae8609 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/1f001470df6748aea71649cffaae8609 2023-08-07 08:53:45,748 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/39fc693feb564d94a22dd0a477a1d4f1 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/39fc693feb564d94a22dd0a477a1d4f1 2023-08-07 08:53:45,749 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/48528e10cbed4040851175be536b961e to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/48528e10cbed4040851175be536b961e 2023-08-07 08:53:45,750 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/4f5f12f2155941219ae22b6ac5110593 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/4f5f12f2155941219ae22b6ac5110593 2023-08-07 08:53:45,751 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/68bc7551705944cb8c4eaeb76c784f25 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/68bc7551705944cb8c4eaeb76c784f25 2023-08-07 08:53:45,751 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/690b4cd18b0d47bea9d964a7c786568b to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/690b4cd18b0d47bea9d964a7c786568b 2023-08-07 08:53:45,752 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/7228be319c5144239680685192b85fb5 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/7228be319c5144239680685192b85fb5 2023-08-07 08:53:45,753 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/876913088cb546d1b4701088081483d7 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/876913088cb546d1b4701088081483d7 2023-08-07 08:53:45,755 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/93712aa9572140b1b56e5833f35671e6 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/93712aa9572140b1b56e5833f35671e6 2023-08-07 08:53:45,756 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/e0d378647ec74ea1b52947731f7efefb to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/cf2/e0d378647ec74ea1b52947731f7efefb 2023-08-07 08:53:45,758 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/recovered.edits/106.seqid to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da/recovered.edits/106.seqid 2023-08-07 08:53:45,758 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0d0fd872f4eb265c927f025c1f8dd4da 2023-08-07 08:53:45,758 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-08-07 08:53:45,760 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:45,762 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-08-07 08:53:45,763 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-08-07 08:53:45,764 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:45,765 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-08-07 08:53:45,765 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691398425765"}]},"ts":"9223372036854775807"} 2023-08-07 08:53:45,765 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691398425765"}]},"ts":"9223372036854775807"} 2023-08-07 08:53:45,765 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691398425765"}]},"ts":"9223372036854775807"} 2023-08-07 08:53:45,766 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-08-07 08:53:45,766 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 0d0fd872f4eb265c927f025c1f8dd4da, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1691398422068.0d0fd872f4eb265c927f025c1f8dd4da.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 890310475de9dc263dd17491ca380a4c, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1691398424234.890310475de9dc263dd17491ca380a4c.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => 246cb6e862f1133bf06f7e32263c91b4, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1691398424234.246cb6e862f1133bf06f7e32263c91b4.', STARTKEY => '60', ENDKEY => ''}] 2023-08-07 08:53:45,766 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-08-07 08:53:45,767 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1691398425767"}]},"ts":"9223372036854775807"} 2023-08-07 08:53:45,768 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-08-07 08:53:45,770 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:53:45,771 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 52 msec 2023-08-07 08:53:45,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=84 2023-08-07 08:53:45,823 INFO [Listener at localhost/33561] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-08-07 08:53:45,838 INFO [Listener at localhost/33561] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=406 (was 401) Potentially hanging thread: hconnection-0x36ae5be-metaLookup-shared--pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1896775705_17 at /127.0.0.1:58806 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1896775705_17 at /127.0.0.1:58898 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1896775705_17 at /127.0.0.1:34924 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x36ae5be-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1896775705_17 at /127.0.0.1:56852 [Waiting for operation #27] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1896775705_17 at /127.0.0.1:33698 [Waiting for operation #49] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1896775705_17 at /127.0.0.1:57448 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1896775705_17 at /127.0.0.1:41420 [Waiting for operation #23] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x36ae5be-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1896775705_17 at /127.0.0.1:34604 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x36ae5be-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1896775705_17 at /127.0.0.1:49126 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1896775705_17 at /127.0.0.1:57054 [Waiting for operation #45] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1896775705_17 at /127.0.0.1:58780 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1896775705_17 at /127.0.0.1:58944 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1896775705_17 at /127.0.0.1:53904 [Waiting for operation #33] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x36ae5be-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x36ae5be-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1896775705_17 at /127.0.0.1:34870 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x36ae5be-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1896775705_17 at /127.0.0.1:49412 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x36ae5be-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x36ae5be-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1896775705_17 at /127.0.0.1:41248 [Waiting for operation #24] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x36ae5be-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=857 (was 855) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=346 (was 368), ProcessCount=174 (was 174), AvailableMemoryMB=1910 (was 1920) 2023-08-07 08:53:45,852 INFO [Listener at localhost/33561] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=406, OpenFileDescriptor=857, MaxFileDescriptor=60000, SystemLoadAverage=346, ProcessCount=174, AvailableMemoryMB=1909 2023-08-07 08:53:45,852 WARN [Listener at localhost/33561] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-07 08:53:45,855 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-08-07 08:53:45,856 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-08-07 08:53:45,857 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:53:45,858 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-08-07 08:53:45,859 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-07 08:53:45,860 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/022268c9253618609543a9a81eddac57 2023-08-07 08:53:45,861 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/022268c9253618609543a9a81eddac57 empty. 2023-08-07 08:53:45,861 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/022268c9253618609543a9a81eddac57 2023-08-07 08:53:45,861 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-08-07 08:53:45,870 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-08-07 08:53:45,871 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7675): creating {ENCODED => 022268c9253618609543a9a81eddac57, NAME => 'testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp 2023-08-07 08:53:45,880 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:53:45,881 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1604): Closing 022268c9253618609543a9a81eddac57, disabling compactions & flushes 2023-08-07 08:53:45,881 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57. 2023-08-07 08:53:45,881 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57. 2023-08-07 08:53:45,881 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57. after waiting 0 ms 2023-08-07 08:53:45,881 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57. 2023-08-07 08:53:45,881 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57. 2023-08-07 08:53:45,881 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1558): Region close journal for 022268c9253618609543a9a81eddac57: 2023-08-07 08:53:45,882 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-08-07 08:53:45,883 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1691398425883"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398425883"}]},"ts":"1691398425883"} 2023-08-07 08:53:45,884 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-07 08:53:45,885 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-07 08:53:45,885 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398425885"}]},"ts":"1691398425885"} 2023-08-07 08:53:45,886 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-08-07 08:53:45,889 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-07 08:53:45,889 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-07 08:53:45,889 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-07 08:53:45,889 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-07 08:53:45,889 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-07 08:53:45,890 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=022268c9253618609543a9a81eddac57, ASSIGN}] 2023-08-07 08:53:45,891 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=022268c9253618609543a9a81eddac57, ASSIGN 2023-08-07 08:53:45,892 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=022268c9253618609543a9a81eddac57, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34633,1691398396427; forceNewPlan=false, retain=false 2023-08-07 08:53:46,042 INFO [jenkins-hbase4:33357] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-07 08:53:46,045 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=022268c9253618609543a9a81eddac57, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:46,045 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1691398426045"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398426045"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398426045"}]},"ts":"1691398426045"} 2023-08-07 08:53:46,047 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure 022268c9253618609543a9a81eddac57, server=jenkins-hbase4.apache.org,34633,1691398396427}] 2023-08-07 08:53:46,199 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:46,199 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-07 08:53:46,203 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36112, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-07 08:53:46,208 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57. 2023-08-07 08:53:46,208 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 022268c9253618609543a9a81eddac57, NAME => 'testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57.', STARTKEY => '', ENDKEY => ''} 2023-08-07 08:53:46,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-07 08:53:46,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 022268c9253618609543a9a81eddac57 2023-08-07 08:53:46,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:53:46,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 022268c9253618609543a9a81eddac57 2023-08-07 08:53:46,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 022268c9253618609543a9a81eddac57 2023-08-07 08:53:46,210 INFO [StoreOpener-022268c9253618609543a9a81eddac57-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 022268c9253618609543a9a81eddac57 2023-08-07 08:53:46,211 DEBUG [StoreOpener-022268c9253618609543a9a81eddac57-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackForSplitTableRegionWithReplica/022268c9253618609543a9a81eddac57/cf1 2023-08-07 08:53:46,211 DEBUG [StoreOpener-022268c9253618609543a9a81eddac57-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackForSplitTableRegionWithReplica/022268c9253618609543a9a81eddac57/cf1 2023-08-07 08:53:46,211 INFO [StoreOpener-022268c9253618609543a9a81eddac57-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 022268c9253618609543a9a81eddac57 columnFamilyName cf1 2023-08-07 08:53:46,212 INFO [StoreOpener-022268c9253618609543a9a81eddac57-1] regionserver.HStore(310): Store=022268c9253618609543a9a81eddac57/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:46,213 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackForSplitTableRegionWithReplica/022268c9253618609543a9a81eddac57 2023-08-07 08:53:46,213 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackForSplitTableRegionWithReplica/022268c9253618609543a9a81eddac57 2023-08-07 08:53:46,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 022268c9253618609543a9a81eddac57 2023-08-07 08:53:46,218 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackForSplitTableRegionWithReplica/022268c9253618609543a9a81eddac57/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 08:53:46,218 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 022268c9253618609543a9a81eddac57; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62414408, jitterRate=-0.06995284557342529}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-07 08:53:46,218 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 022268c9253618609543a9a81eddac57: 2023-08-07 08:53:46,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57., pid=87, masterSystemTime=1691398426199 2023-08-07 08:53:46,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57. 2023-08-07 08:53:46,222 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57. 2023-08-07 08:53:46,222 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=022268c9253618609543a9a81eddac57, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:46,222 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1691398426222"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691398426222"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691398426222"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398426222"}]},"ts":"1691398426222"} 2023-08-07 08:53:46,225 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-08-07 08:53:46,225 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure 022268c9253618609543a9a81eddac57, server=jenkins-hbase4.apache.org,34633,1691398396427 in 176 msec 2023-08-07 08:53:46,226 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-08-07 08:53:46,226 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=022268c9253618609543a9a81eddac57, ASSIGN in 335 msec 2023-08-07 08:53:46,227 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-07 08:53:46,227 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398426227"}]},"ts":"1691398426227"} 2023-08-07 08:53:46,228 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-08-07 08:53:46,233 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-08-07 08:53:46,234 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 377 msec 2023-08-07 08:53:46,366 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-07 08:53:46,373 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:53:46,386 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-08-07 08:53:46,387 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-08-07 08:53:46,387 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-08-07 08:53:46,390 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-08-07 08:53:46,401 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=022268c9253618609543a9a81eddac57, REOPEN/MOVE}] 2023-08-07 08:53:46,402 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=022268c9253618609543a9a81eddac57, REOPEN/MOVE 2023-08-07 08:53:46,403 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=022268c9253618609543a9a81eddac57, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:46,403 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1691398426403"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398426403"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398426403"}]},"ts":"1691398426403"} 2023-08-07 08:53:46,404 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure 022268c9253618609543a9a81eddac57, server=jenkins-hbase4.apache.org,34633,1691398396427}] 2023-08-07 08:53:46,556 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 022268c9253618609543a9a81eddac57 2023-08-07 08:53:46,557 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 022268c9253618609543a9a81eddac57, disabling compactions & flushes 2023-08-07 08:53:46,557 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57. 2023-08-07 08:53:46,557 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57. 2023-08-07 08:53:46,558 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57. after waiting 0 ms 2023-08-07 08:53:46,558 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57. 2023-08-07 08:53:46,561 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackForSplitTableRegionWithReplica/022268c9253618609543a9a81eddac57/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-08-07 08:53:46,561 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-07 08:53:46,561 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57. 2023-08-07 08:53:46,562 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 022268c9253618609543a9a81eddac57: 2023-08-07 08:53:46,562 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3510): Not adding moved region record: 022268c9253618609543a9a81eddac57 to self. 2023-08-07 08:53:46,563 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 022268c9253618609543a9a81eddac57 2023-08-07 08:53:46,563 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=022268c9253618609543a9a81eddac57, regionState=CLOSED 2023-08-07 08:53:46,563 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1691398426563"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398426563"}]},"ts":"1691398426563"} 2023-08-07 08:53:46,566 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=90 2023-08-07 08:53:46,566 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure 022268c9253618609543a9a81eddac57, server=jenkins-hbase4.apache.org,34633,1691398396427 in 160 msec 2023-08-07 08:53:46,567 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=022268c9253618609543a9a81eddac57, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,34633,1691398396427; forceNewPlan=false, retain=true 2023-08-07 08:53:46,717 INFO [jenkins-hbase4:33357] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-07 08:53:46,717 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=022268c9253618609543a9a81eddac57, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:46,717 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1691398426717"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398426717"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398426717"}]},"ts":"1691398426717"} 2023-08-07 08:53:46,719 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure 022268c9253618609543a9a81eddac57, server=jenkins-hbase4.apache.org,34633,1691398396427}] 2023-08-07 08:53:46,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57. 2023-08-07 08:53:46,873 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 022268c9253618609543a9a81eddac57, NAME => 'testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57.', STARTKEY => '', ENDKEY => ''} 2023-08-07 08:53:46,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-07 08:53:46,874 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 022268c9253618609543a9a81eddac57 2023-08-07 08:53:46,874 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:53:46,874 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 022268c9253618609543a9a81eddac57 2023-08-07 08:53:46,874 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 022268c9253618609543a9a81eddac57 2023-08-07 08:53:46,875 INFO [StoreOpener-022268c9253618609543a9a81eddac57-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 022268c9253618609543a9a81eddac57 2023-08-07 08:53:46,876 DEBUG [StoreOpener-022268c9253618609543a9a81eddac57-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackForSplitTableRegionWithReplica/022268c9253618609543a9a81eddac57/cf1 2023-08-07 08:53:46,876 DEBUG [StoreOpener-022268c9253618609543a9a81eddac57-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackForSplitTableRegionWithReplica/022268c9253618609543a9a81eddac57/cf1 2023-08-07 08:53:46,876 INFO [StoreOpener-022268c9253618609543a9a81eddac57-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 022268c9253618609543a9a81eddac57 columnFamilyName cf1 2023-08-07 08:53:46,876 INFO [StoreOpener-022268c9253618609543a9a81eddac57-1] regionserver.HStore(310): Store=022268c9253618609543a9a81eddac57/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:53:46,877 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackForSplitTableRegionWithReplica/022268c9253618609543a9a81eddac57 2023-08-07 08:53:46,878 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackForSplitTableRegionWithReplica/022268c9253618609543a9a81eddac57 2023-08-07 08:53:46,881 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 022268c9253618609543a9a81eddac57 2023-08-07 08:53:46,881 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 022268c9253618609543a9a81eddac57; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63227025, jitterRate=-0.05784390866756439}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-07 08:53:46,881 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 022268c9253618609543a9a81eddac57: 2023-08-07 08:53:46,882 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57., pid=92, masterSystemTime=1691398426870 2023-08-07 08:53:46,883 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57. 2023-08-07 08:53:46,883 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57. 2023-08-07 08:53:46,884 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=022268c9253618609543a9a81eddac57, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:53:46,884 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1691398426884"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691398426884"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691398426884"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398426884"}]},"ts":"1691398426884"} 2023-08-07 08:53:46,886 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-08-07 08:53:46,887 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure 022268c9253618609543a9a81eddac57, server=jenkins-hbase4.apache.org,34633,1691398396427 in 166 msec 2023-08-07 08:53:46,888 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-08-07 08:53:46,888 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=022268c9253618609543a9a81eddac57, REOPEN/MOVE in 486 msec 2023-08-07 08:53:46,891 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-08-07 08:53:46,891 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 500 msec 2023-08-07 08:53:46,895 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=b0ddf60071e752aa19c5e065b70504c9, ASSIGN}] 2023-08-07 08:53:46,896 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=b0ddf60071e752aa19c5e065b70504c9, ASSIGN 2023-08-07 08:53:46,897 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=b0ddf60071e752aa19c5e065b70504c9, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-08-07 08:53:47,047 DEBUG [jenkins-hbase4:33357] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-07 08:53:47,048 DEBUG [jenkins-hbase4:33357] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-07 08:53:47,048 DEBUG [jenkins-hbase4:33357] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-07 08:53:47,048 DEBUG [jenkins-hbase4:33357] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-07 08:53:47,048 DEBUG [jenkins-hbase4:33357] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-08-07 08:53:47,050 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=b0ddf60071e752aa19c5e065b70504c9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:53:47,050 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1691398427050"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1691398427050"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1691398427050"}]},"ts":"1691398427050"} 2023-08-07 08:53:47,054 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure b0ddf60071e752aa19c5e065b70504c9, server=jenkins-hbase4.apache.org,34559,1691398396467}] 2023-08-07 08:53:47,208 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackForSplitTableRegionWithReplica,,1691398425856_0001.b0ddf60071e752aa19c5e065b70504c9. 2023-08-07 08:53:47,208 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b0ddf60071e752aa19c5e065b70504c9, NAME => 'testRollbackForSplitTableRegionWithReplica,,1691398425856_0001.b0ddf60071e752aa19c5e065b70504c9.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-08-07 08:53:47,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-07 08:53:47,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica b0ddf60071e752aa19c5e065b70504c9 2023-08-07 08:53:47,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1691398425856_0001.b0ddf60071e752aa19c5e065b70504c9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:53:47,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b0ddf60071e752aa19c5e065b70504c9 2023-08-07 08:53:47,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b0ddf60071e752aa19c5e065b70504c9 2023-08-07 08:53:47,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:47,309 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:47,410 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:47,510 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:47,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:47,710 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:47,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:47,842 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-08-07 08:53:47,911 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:48,011 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:48,111 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:48,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:48,311 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:48,412 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:48,512 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:48,612 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:48,712 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:48,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:48,913 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:49,013 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:49,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:49,213 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:49,313 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:49,414 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:49,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:49,614 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:49,714 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:49,814 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:49,915 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:49,934 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-08-07 08:53:50,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:50,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:50,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:50,315 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:50,416 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:50,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:50,616 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:50,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:50,816 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:50,917 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:51,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:51,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:51,217 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:51,317 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:51,417 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:51,518 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:51,618 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:51,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:51,818 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:51,918 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:52,019 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:52,119 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:52,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:52,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:52,419 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:52,520 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:52,620 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:52,720 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:52,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:52,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:53,021 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:53,121 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:53,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:53,321 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:53,421 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:53,522 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:53,622 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:53,722 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:53,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:53,922 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:54,023 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:54,123 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:54,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:54,323 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:54,423 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:54,524 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:54,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:54,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:54,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:54,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:55,025 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:55,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:55,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:55,325 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:55,425 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:55,526 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:55,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:55,726 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:55,826 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:55,927 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:56,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:56,127 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:56,227 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:56,327 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:56,428 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:56,528 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:56,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:56,728 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:56,828 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:56,929 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:57,029 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:57,129 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:57,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:57,329 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:57,430 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:57,530 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:57,630 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:57,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:57,830 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:57,931 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:58,031 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:58,131 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:58,231 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:58,331 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:58,432 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:58,532 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:58,632 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:58,732 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:58,833 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:58,933 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:59,033 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:59,133 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:59,233 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:59,334 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:59,434 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:59,534 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:59,634 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:59,734 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:59,835 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:53:59,935 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:00,035 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:00,135 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:00,235 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:00,336 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:00,436 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:00,536 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:00,636 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:00,736 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:00,837 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:00,937 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:01,037 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:01,137 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:01,237 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:01,338 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:01,438 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:01,538 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:01,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:01,738 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:01,839 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:01,939 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:02,039 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:02,139 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:02,239 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:02,340 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:02,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:02,540 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:02,640 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:02,740 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:02,841 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:02,941 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:03,041 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:03,141 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:03,242 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:03,342 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:03,442 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:03,542 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:03,643 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:03,743 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:03,843 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:03,943 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:04,043 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:04,143 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:04,244 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:04,344 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:04,444 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:04,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:04,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:04,745 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:04,845 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:04,945 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:05,045 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:05,145 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:05,246 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:05,346 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:05,446 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:05,546 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:05,646 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:05,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:05,847 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:05,947 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:06,047 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:06,148 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:06,248 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:06,348 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:06,448 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:06,548 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:06,649 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:06,749 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:06,849 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:06,949 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:07,049 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:07,150 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:07,250 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:07,350 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:07,450 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:07,550 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:07,651 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:07,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:07,851 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:07,951 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:08,051 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:08,151 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:08,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:08,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:08,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:08,552 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:08,652 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:08,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:08,853 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:08,953 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:09,053 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:09,153 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:09,254 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:09,354 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:09,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:09,554 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:09,655 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:09,755 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:09,855 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:09,955 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:10,055 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:10,155 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:10,256 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:10,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:10,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:10,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:10,656 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:10,757 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:10,857 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:10,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:11,057 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:11,157 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:11,258 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:11,358 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:11,458 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:11,558 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:11,658 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:11,759 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:11,859 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:11,959 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:12,059 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:12,159 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:12,260 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:12,360 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:12,460 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:12,560 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:12,661 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:12,761 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:12,861 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:12,961 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:13,062 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:13,162 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:13,262 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:13,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:13,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:13,563 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:13,663 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:13,763 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:13,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:13,963 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:14,064 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:14,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:14,264 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:14,364 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:14,464 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:14,565 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:14,665 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:14,765 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:14,865 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:14,965 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:15,066 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:15,166 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:15,266 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:15,366 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:15,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:15,567 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:15,667 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:15,767 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:15,867 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:15,967 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:16,068 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:16,168 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:16,268 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:16,368 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:16,469 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:16,569 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:16,669 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:16,769 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:16,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:16,969 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:17,070 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:17,170 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:17,270 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:17,370 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:17,470 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:17,571 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:17,671 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:17,771 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:17,871 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:17,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:18,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:18,172 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:18,272 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:18,372 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:18,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:18,573 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:18,673 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:18,773 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:18,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:18,974 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:19,074 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:19,174 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:19,274 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:19,374 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:19,474 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:19,575 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:19,675 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:19,775 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:19,875 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:19,975 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:20,076 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:20,176 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:20,276 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:20,376 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:20,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:20,577 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:20,677 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:20,777 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:20,877 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:20,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:21,078 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:21,178 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:21,278 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:21,378 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:21,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:21,579 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:21,679 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:21,779 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:21,879 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:21,980 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:22,080 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:22,180 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:22,280 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:22,380 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:22,481 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:22,581 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:22,681 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:22,781 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:22,881 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:22,982 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:23,082 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:23,182 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:23,282 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:23,382 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:23,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:23,583 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:23,683 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:23,783 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:23,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:23,984 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:24,084 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:24,184 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:24,284 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:24,385 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:24,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:24,585 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:24,685 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:24,785 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:24,886 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:24,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:25,086 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:25,186 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:25,287 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:25,387 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:25,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:25,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:25,687 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:25,788 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:25,888 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:25,988 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:26,088 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:26,188 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:26,289 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:26,389 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:26,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:26,589 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:26,689 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:26,789 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:26,890 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:26,990 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:27,090 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:27,190 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:27,290 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:27,391 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:27,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:27,591 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:27,691 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:27,791 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:27,892 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:27,992 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:28,092 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:28,192 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:28,293 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:28,393 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:28,493 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:28,593 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:28,694 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:28,794 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:28,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:28,994 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:29,095 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:29,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:29,295 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:29,395 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:29,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:29,596 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:29,696 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:29,796 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:29,896 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:29,996 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:30,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:30,197 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:30,297 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:30,397 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:30,497 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:30,598 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:30,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:30,798 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:30,899 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:30,999 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:31,099 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:31,199 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:31,300 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:31,400 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:31,500 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:31,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:31,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:31,801 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:31,901 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:32,001 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:32,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:32,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:32,302 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:32,402 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:32,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:32,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:32,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:32,802 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:32,903 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:33,003 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:33,103 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:33,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:33,304 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:33,404 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:33,504 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:33,604 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:33,704 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:33,805 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:33,905 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:34,005 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:34,105 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:34,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:34,306 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:34,406 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:34,506 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:34,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:34,707 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:34,807 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:34,907 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:35,007 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:35,107 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:35,208 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:35,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:35,408 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:35,508 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:35,608 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:35,709 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:35,809 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:35,909 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:36,009 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:36,110 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:36,210 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:36,310 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:36,410 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:36,510 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:36,611 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:36,711 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:36,811 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:36,911 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:37,011 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:37,112 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:37,212 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:37,312 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:37,412 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:37,513 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:37,613 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:37,713 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:37,813 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:37,913 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:38,014 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:38,114 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:38,214 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:38,314 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:38,414 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:38,515 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:38,615 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:38,715 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:38,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:38,915 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:39,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:39,116 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:39,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:39,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:39,416 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:39,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:39,617 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:39,717 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:39,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:39,917 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:40,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:40,118 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:40,218 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:40,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:40,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:40,519 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:40,619 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:40,719 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:40,819 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:40,919 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:41,020 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:41,120 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:41,220 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:41,320 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:41,420 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:41,521 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:41,621 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:41,721 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:41,821 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:41,921 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:42,022 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:42,122 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:42,222 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:42,322 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:42,422 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:42,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:42,623 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:42,723 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:42,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:42,923 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:43,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:43,124 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:43,224 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:43,324 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:43,424 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:43,525 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:43,625 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:43,725 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:43,825 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:43,925 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:44,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:44,126 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:44,226 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:44,326 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:44,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:44,527 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:44,627 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:44,727 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:44,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:44,928 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:45,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:45,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:45,228 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:45,328 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:45,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:45,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:45,629 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:45,729 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:45,829 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:45,930 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:46,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:46,130 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:46,230 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:46,330 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:46,431 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:46,480 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-08-07 08:54:46,531 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:46,581 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-08-07 08:54:46,631 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:46,731 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:46,782 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-08-07 08:54:46,831 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:46,932 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:47,032 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:47,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-08-07 08:54:47,132 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:47,232 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:47,332 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:47,433 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:47,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:47,584 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-08-07 08:54:47,633 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:47,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:47,833 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:47,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:48,034 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:48,134 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:48,234 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:48,334 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:48,435 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:48,535 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:48,586 DEBUG [Listener at localhost/33561] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:48,587 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=022268c9253618609543a9a81eddac57, daughterA=3088038bd36f4f725bfab047b7dd3b57, daughterB=60ab8c2a890a3fc5d5fda3e377b4c31f 2023-08-07 08:54:48,588 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=022268c9253618609543a9a81eddac57, daughterA=3088038bd36f4f725bfab047b7dd3b57, daughterB=60ab8c2a890a3fc5d5fda3e377b4c31f 2023-08-07 08:54:48,588 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=022268c9253618609543a9a81eddac57, daughterA=3088038bd36f4f725bfab047b7dd3b57, daughterB=60ab8c2a890a3fc5d5fda3e377b4c31f 2023-08-07 08:54:48,588 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=022268c9253618609543a9a81eddac57, daughterA=3088038bd36f4f725bfab047b7dd3b57, daughterB=60ab8c2a890a3fc5d5fda3e377b4c31f 2023-08-07 08:54:48,590 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting 022268c9253618609543a9a81eddac57, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=022268c9253618609543a9a81eddac57, daughterA=3088038bd36f4f725bfab047b7dd3b57, daughterB=60ab8c2a890a3fc5d5fda3e377b4c31f org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,34559,1691398396467, table=testRollbackForSplitTableRegionWithReplica, region=b0ddf60071e752aa19c5e065b70504c9 is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:54:48,593 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,34559,1691398396467, table=testRollbackForSplitTableRegionWithReplica, region=b0ddf60071e752aa19c5e065b70504c9 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=022268c9253618609543a9a81eddac57, daughterA=3088038bd36f4f725bfab047b7dd3b57, daughterB=60ab8c2a890a3fc5d5fda3e377b4c31f exec-time=6 msec 2023-08-07 08:54:48,635 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:48,735 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:48,835 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-07 08:54:48,936 INFO [StoreOpener-b0ddf60071e752aa19c5e065b70504c9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b0ddf60071e752aa19c5e065b70504c9 2023-08-07 08:54:48,939 DEBUG [StoreOpener-b0ddf60071e752aa19c5e065b70504c9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackForSplitTableRegionWithReplica/022268c9253618609543a9a81eddac57/cf1 2023-08-07 08:54:48,939 DEBUG [StoreOpener-b0ddf60071e752aa19c5e065b70504c9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackForSplitTableRegionWithReplica/022268c9253618609543a9a81eddac57/cf1 2023-08-07 08:54:48,939 INFO [StoreOpener-b0ddf60071e752aa19c5e065b70504c9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b0ddf60071e752aa19c5e065b70504c9 columnFamilyName cf1 2023-08-07 08:54:48,940 INFO [StoreOpener-b0ddf60071e752aa19c5e065b70504c9-1] regionserver.HStore(310): Store=b0ddf60071e752aa19c5e065b70504c9/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:54:48,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b0ddf60071e752aa19c5e065b70504c9; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64246457, jitterRate=-0.04265318810939789}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-07 08:54:48,942 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b0ddf60071e752aa19c5e065b70504c9: 2023-08-07 08:54:48,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1691398425856_0001.b0ddf60071e752aa19c5e065b70504c9., pid=94, masterSystemTime=1691398427205 2023-08-07 08:54:48,943 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1691398425856_0001.b0ddf60071e752aa19c5e065b70504c9. 2023-08-07 08:54:48,943 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackForSplitTableRegionWithReplica,,1691398425856_0001.b0ddf60071e752aa19c5e065b70504c9. 2023-08-07 08:54:48,944 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=b0ddf60071e752aa19c5e065b70504c9, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:48,944 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1691398488944"},{"qualifier":"server_0001","vlen":31,"tag":[],"timestamp":"1691398488944"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1691398488944"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1691398488944"}]},"ts":"1691398488944"} 2023-08-07 08:54:48,947 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-08-07 08:54:48,947 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure b0ddf60071e752aa19c5e065b70504c9, server=jenkins-hbase4.apache.org,34559,1691398396467 in 1 mins, 1.893 sec 2023-08-07 08:54:48,948 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=88 2023-08-07 08:54:48,949 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=b0ddf60071e752aa19c5e065b70504c9, ASSIGN in 1 mins, 2.052 sec 2023-08-07 08:54:48,955 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.58 sec 2023-08-07 08:54:48,955 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1691398426363","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62592,"client":"172.31.14.131:48412","queuetimems":0,"class":"HMaster"} 2023-08-07 08:54:48,956 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(156): Unknown callId: 603, skipping over this response of 3 bytes 2023-08-07 08:54:50,838 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,34559,1691398396467, table=testRollbackForSplitTableRegionWithReplica, region=b0ddf60071e752aa19c5e065b70504c9 is currently in transition, give up 2023-08-07 08:54:50,840 WARN [Listener at localhost/33561] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-07 08:54:50,841 INFO [Listener at localhost/33561] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:50,842 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:50,843 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:50,845 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-08-07 08:54:50,845 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398490845"}]},"ts":"1691398490845"} 2023-08-07 08:54:50,847 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-08-07 08:54:50,848 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-08-07 08:54:50,849 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=022268c9253618609543a9a81eddac57, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=b0ddf60071e752aa19c5e065b70504c9, UNASSIGN}] 2023-08-07 08:54:50,849 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=b0ddf60071e752aa19c5e065b70504c9, UNASSIGN 2023-08-07 08:54:50,850 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=b0ddf60071e752aa19c5e065b70504c9, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:50,850 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1691398490850"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1691398490850"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1691398490850"}]},"ts":"1691398490850"} 2023-08-07 08:54:50,852 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure b0ddf60071e752aa19c5e065b70504c9, server=jenkins-hbase4.apache.org,34559,1691398396467}] 2023-08-07 08:54:50,854 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=022268c9253618609543a9a81eddac57, UNASSIGN 2023-08-07 08:54:50,854 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=022268c9253618609543a9a81eddac57, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:50,855 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1691398490854"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398490854"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398490854"}]},"ts":"1691398490854"} 2023-08-07 08:54:50,856 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure 022268c9253618609543a9a81eddac57, server=jenkins-hbase4.apache.org,34633,1691398396427}] 2023-08-07 08:54:50,946 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-08-07 08:54:51,004 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close b0ddf60071e752aa19c5e065b70504c9 2023-08-07 08:54:51,005 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b0ddf60071e752aa19c5e065b70504c9, disabling compactions & flushes 2023-08-07 08:54:51,005 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1691398425856_0001.b0ddf60071e752aa19c5e065b70504c9. 2023-08-07 08:54:51,005 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1691398425856_0001.b0ddf60071e752aa19c5e065b70504c9. 2023-08-07 08:54:51,005 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1691398425856_0001.b0ddf60071e752aa19c5e065b70504c9. after waiting 0 ms 2023-08-07 08:54:51,005 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1691398425856_0001.b0ddf60071e752aa19c5e065b70504c9. 2023-08-07 08:54:51,005 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-07 08:54:51,006 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1691398425856_0001.b0ddf60071e752aa19c5e065b70504c9. 2023-08-07 08:54:51,006 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b0ddf60071e752aa19c5e065b70504c9: 2023-08-07 08:54:51,008 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed b0ddf60071e752aa19c5e065b70504c9 2023-08-07 08:54:51,008 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 022268c9253618609543a9a81eddac57 2023-08-07 08:54:51,008 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 022268c9253618609543a9a81eddac57, disabling compactions & flushes 2023-08-07 08:54:51,009 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57. 2023-08-07 08:54:51,009 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57. 2023-08-07 08:54:51,009 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57. after waiting 0 ms 2023-08-07 08:54:51,009 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57. 2023-08-07 08:54:51,009 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=b0ddf60071e752aa19c5e065b70504c9, regionState=CLOSED 2023-08-07 08:54:51,009 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1691398491009"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1691398491009"}]},"ts":"1691398491009"} 2023-08-07 08:54:51,013 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-08-07 08:54:51,013 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRollbackForSplitTableRegionWithReplica/022268c9253618609543a9a81eddac57/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-08-07 08:54:51,013 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure b0ddf60071e752aa19c5e065b70504c9, server=jenkins-hbase4.apache.org,34559,1691398396467 in 159 msec 2023-08-07 08:54:51,014 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-07 08:54:51,014 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57. 2023-08-07 08:54:51,014 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 022268c9253618609543a9a81eddac57: 2023-08-07 08:54:51,015 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=b0ddf60071e752aa19c5e065b70504c9, UNASSIGN in 164 msec 2023-08-07 08:54:51,016 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 022268c9253618609543a9a81eddac57 2023-08-07 08:54:51,016 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=022268c9253618609543a9a81eddac57, regionState=CLOSED 2023-08-07 08:54:51,016 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1691398491016"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398491016"}]},"ts":"1691398491016"} 2023-08-07 08:54:51,018 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-08-07 08:54:51,018 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure 022268c9253618609543a9a81eddac57, server=jenkins-hbase4.apache.org,34633,1691398396427 in 161 msec 2023-08-07 08:54:51,020 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-08-07 08:54:51,020 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=022268c9253618609543a9a81eddac57, UNASSIGN in 170 msec 2023-08-07 08:54:51,021 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398491021"}]},"ts":"1691398491021"} 2023-08-07 08:54:51,022 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-08-07 08:54:51,023 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-08-07 08:54:51,025 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 181 msec 2023-08-07 08:54:51,147 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-08-07 08:54:51,148 INFO [Listener at localhost/33561] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-08-07 08:54:51,148 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:51,149 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:51,151 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:51,151 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:51,151 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=101 2023-08-07 08:54:51,156 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/022268c9253618609543a9a81eddac57 2023-08-07 08:54:51,158 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/022268c9253618609543a9a81eddac57/cf1, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/022268c9253618609543a9a81eddac57/recovered.edits] 2023-08-07 08:54:51,163 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/022268c9253618609543a9a81eddac57/recovered.edits/7.seqid to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRollbackForSplitTableRegionWithReplica/022268c9253618609543a9a81eddac57/recovered.edits/7.seqid 2023-08-07 08:54:51,163 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/022268c9253618609543a9a81eddac57 2023-08-07 08:54:51,163 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-08-07 08:54:51,166 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:51,168 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-08-07 08:54:51,170 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-08-07 08:54:51,172 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:51,172 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-08-07 08:54:51,172 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691398491172"}]},"ts":"9223372036854775807"} 2023-08-07 08:54:51,172 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1691398425856_0001.b0ddf60071e752aa19c5e065b70504c9.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691398491172"}]},"ts":"9223372036854775807"} 2023-08-07 08:54:51,174 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-08-07 08:54:51,174 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 022268c9253618609543a9a81eddac57, NAME => 'testRollbackForSplitTableRegionWithReplica,,1691398425856.022268c9253618609543a9a81eddac57.', STARTKEY => '', ENDKEY => ''}, {ENCODED => b0ddf60071e752aa19c5e065b70504c9, NAME => 'testRollbackForSplitTableRegionWithReplica,,1691398425856_0001.b0ddf60071e752aa19c5e065b70504c9.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-08-07 08:54:51,174 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-08-07 08:54:51,174 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1691398491174"}]},"ts":"9223372036854775807"} 2023-08-07 08:54:51,176 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-08-07 08:54:51,182 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:51,183 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 34 msec 2023-08-07 08:54:51,253 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=101 2023-08-07 08:54:51,254 INFO [Listener at localhost/33561] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-08-07 08:54:51,268 INFO [Listener at localhost/33561] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=347 (was 406), OpenFileDescriptor=659 (was 857), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=202 (was 346), ProcessCount=175 (was 174) - ProcessCount LEAK? -, AvailableMemoryMB=1898 (was 1909) 2023-08-07 08:54:51,281 INFO [Listener at localhost/33561] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=348, OpenFileDescriptor=659, MaxFileDescriptor=60000, SystemLoadAverage=202, ProcessCount=175, AvailableMemoryMB=1897 2023-08-07 08:54:51,281 WARN [Listener at localhost/33561] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-07 08:54:51,284 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-08-07 08:54:51,284 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-08-07 08:54:51,286 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(1029): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:51,287 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-08-07 08:54:51,288 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-07 08:54:51,290 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f 2023-08-07 08:54:51,290 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f empty. 2023-08-07 08:54:51,291 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f 2023-08-07 08:54:51,291 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-08-07 08:54:51,306 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-08-07 08:54:51,307 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7675): creating {ENCODED => c3321c1fa2ec0c6f32bbed977602bc2f, NAME => 'testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp 2023-08-07 08:54:51,321 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(866): Instantiated testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:54:51,321 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1604): Closing c3321c1fa2ec0c6f32bbed977602bc2f, disabling compactions & flushes 2023-08-07 08:54:51,321 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1626): Closing region testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f. 2023-08-07 08:54:51,321 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f. 2023-08-07 08:54:51,321 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1714): Acquired close lock on testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f. after waiting 0 ms 2023-08-07 08:54:51,321 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1724): Updates disabled for region testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f. 2023-08-07 08:54:51,321 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1838): Closed testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f. 2023-08-07 08:54:51,321 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1558): Region close journal for c3321c1fa2ec0c6f32bbed977602bc2f: 2023-08-07 08:54:51,323 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-08-07 08:54:51,324 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1691398491324"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398491324"}]},"ts":"1691398491324"} 2023-08-07 08:54:51,325 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-07 08:54:51,326 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-07 08:54:51,326 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398491326"}]},"ts":"1691398491326"} 2023-08-07 08:54:51,327 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-08-07 08:54:51,330 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-07 08:54:51,331 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-07 08:54:51,331 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-07 08:54:51,331 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-07 08:54:51,331 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-07 08:54:51,331 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=c3321c1fa2ec0c6f32bbed977602bc2f, ASSIGN}] 2023-08-07 08:54:51,333 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=c3321c1fa2ec0c6f32bbed977602bc2f, ASSIGN 2023-08-07 08:54:51,333 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=c3321c1fa2ec0c6f32bbed977602bc2f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34559,1691398396467; forceNewPlan=false, retain=false 2023-08-07 08:54:51,484 INFO [jenkins-hbase4:33357] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-07 08:54:51,485 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=c3321c1fa2ec0c6f32bbed977602bc2f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:51,485 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1691398491485"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398491485"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398491485"}]},"ts":"1691398491485"} 2023-08-07 08:54:51,487 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure c3321c1fa2ec0c6f32bbed977602bc2f, server=jenkins-hbase4.apache.org,34559,1691398396467}] 2023-08-07 08:54:51,642 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f. 2023-08-07 08:54:51,642 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c3321c1fa2ec0c6f32bbed977602bc2f, NAME => 'testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f.', STARTKEY => '', ENDKEY => ''} 2023-08-07 08:54:51,642 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-07 08:54:51,642 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey c3321c1fa2ec0c6f32bbed977602bc2f 2023-08-07 08:54:51,642 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:54:51,642 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for c3321c1fa2ec0c6f32bbed977602bc2f 2023-08-07 08:54:51,642 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for c3321c1fa2ec0c6f32bbed977602bc2f 2023-08-07 08:54:51,644 INFO [StoreOpener-c3321c1fa2ec0c6f32bbed977602bc2f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c3321c1fa2ec0c6f32bbed977602bc2f 2023-08-07 08:54:51,645 DEBUG [StoreOpener-c3321c1fa2ec0c6f32bbed977602bc2f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf1 2023-08-07 08:54:51,645 DEBUG [StoreOpener-c3321c1fa2ec0c6f32bbed977602bc2f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf1 2023-08-07 08:54:51,645 INFO [StoreOpener-c3321c1fa2ec0c6f32bbed977602bc2f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c3321c1fa2ec0c6f32bbed977602bc2f columnFamilyName cf1 2023-08-07 08:54:51,646 INFO [StoreOpener-c3321c1fa2ec0c6f32bbed977602bc2f-1] regionserver.HStore(310): Store=c3321c1fa2ec0c6f32bbed977602bc2f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:54:51,646 INFO [StoreOpener-c3321c1fa2ec0c6f32bbed977602bc2f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c3321c1fa2ec0c6f32bbed977602bc2f 2023-08-07 08:54:51,647 DEBUG [StoreOpener-c3321c1fa2ec0c6f32bbed977602bc2f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf2 2023-08-07 08:54:51,647 DEBUG [StoreOpener-c3321c1fa2ec0c6f32bbed977602bc2f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf2 2023-08-07 08:54:51,648 INFO [StoreOpener-c3321c1fa2ec0c6f32bbed977602bc2f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c3321c1fa2ec0c6f32bbed977602bc2f columnFamilyName cf2 2023-08-07 08:54:51,648 INFO [StoreOpener-c3321c1fa2ec0c6f32bbed977602bc2f-1] regionserver.HStore(310): Store=c3321c1fa2ec0c6f32bbed977602bc2f/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:54:51,649 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f 2023-08-07 08:54:51,649 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f 2023-08-07 08:54:51,651 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-07 08:54:51,652 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for c3321c1fa2ec0c6f32bbed977602bc2f 2023-08-07 08:54:51,653 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 08:54:51,654 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened c3321c1fa2ec0c6f32bbed977602bc2f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62913686, jitterRate=-0.06251302361488342}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-07 08:54:51,654 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for c3321c1fa2ec0c6f32bbed977602bc2f: 2023-08-07 08:54:51,654 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f., pid=104, masterSystemTime=1691398491638 2023-08-07 08:54:51,656 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f. 2023-08-07 08:54:51,656 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f. 2023-08-07 08:54:51,656 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=c3321c1fa2ec0c6f32bbed977602bc2f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:51,656 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1691398491656"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691398491656"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691398491656"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398491656"}]},"ts":"1691398491656"} 2023-08-07 08:54:51,659 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=104, resume processing ppid=103 2023-08-07 08:54:51,659 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure c3321c1fa2ec0c6f32bbed977602bc2f, server=jenkins-hbase4.apache.org,34559,1691398396467 in 171 msec 2023-08-07 08:54:51,661 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-08-07 08:54:51,661 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=c3321c1fa2ec0c6f32bbed977602bc2f, ASSIGN in 328 msec 2023-08-07 08:54:51,662 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-07 08:54:51,662 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398491662"}]},"ts":"1691398491662"} 2023-08-07 08:54:51,663 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-08-07 08:54:51,665 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-08-07 08:54:51,666 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 380 msec 2023-08-07 08:54:51,794 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:54:51,796 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-08-07 08:54:51,796 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-08-07 08:54:51,796 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:51,796 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-08-07 08:54:51,796 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-08-07 08:54:51,796 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:51,796 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:51,799 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:51,799 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:51,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:51,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:51,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:51,799 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:51,799 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:51,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:51,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:51,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:51,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:51,799 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:51,799 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:54:51,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:51,800 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:51,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:51,800 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:51,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:54:51,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:51,800 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:51,800 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-07 08:54:51,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:54:51,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:51,800 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-07 08:54:51,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:54:51,800 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-07 08:54:51,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:51,800 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-07 08:54:51,800 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-07 08:54:51,800 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:51,800 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-07 08:54:51,801 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-07 08:54:51,801 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-07 08:54:51,801 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-07 08:54:51,801 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-07 08:54:51,801 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-07 08:54:51,801 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-07 08:54:51,801 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:51,801 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:51,801 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-07 08:54:51,801 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-07 08:54:51,801 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-07 08:54:51,801 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-07 08:54:51,801 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:54:51,801 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:54:51,801 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f. 2023-08-07 08:54:51,801 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f. started... 2023-08-07 08:54:51,802 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c3321c1fa2ec0c6f32bbed977602bc2f 2/2 column families, dataSize=120 B heapSize=928 B 2023-08-07 08:54:51,802 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:51,803 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:51,803 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:51,803 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:54:51,803 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:51,803 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:54:51,813 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/.tmp/cf1/db4278d19b3b46b99d03e79749307867 2023-08-07 08:54:51,829 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/.tmp/cf2/44cfd9773a9e422a93d5d0e31ad82f94 2023-08-07 08:54:51,834 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/.tmp/cf1/db4278d19b3b46b99d03e79749307867 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf1/db4278d19b3b46b99d03e79749307867 2023-08-07 08:54:51,838 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf1/db4278d19b3b46b99d03e79749307867, entries=2, sequenceid=6, filesize=4.8 K 2023-08-07 08:54:51,839 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/.tmp/cf2/44cfd9773a9e422a93d5d0e31ad82f94 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf2/44cfd9773a9e422a93d5d0e31ad82f94 2023-08-07 08:54:51,843 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf2/44cfd9773a9e422a93d5d0e31ad82f94, entries=2, sequenceid=6, filesize=4.8 K 2023-08-07 08:54:51,844 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for c3321c1fa2ec0c6f32bbed977602bc2f in 42ms, sequenceid=6, compaction requested=false 2023-08-07 08:54:51,844 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-08-07 08:54:51,845 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c3321c1fa2ec0c6f32bbed977602bc2f: 2023-08-07 08:54:51,846 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f. 2023-08-07 08:54:51,846 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:54:51,846 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:54:51,846 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:51,846 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-07 08:54:51,846 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-07 08:54:51,847 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:51,847 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:51,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:51,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:54:51,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:54:51,848 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:51,848 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:54:51,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:54:51,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:54:51,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-07 08:54:51,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:51,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:51,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:51,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:54:51,850 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-08-07 08:54:51,850 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@e4feb8e[Count = 0] remaining members to acquire global barrier 2023-08-07 08:54:51,850 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-08-07 08:54:51,850 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:51,852 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:51,852 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:51,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:51,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:51,852 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:51,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:51,852 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:51,852 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-07 08:54:51,853 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-07 08:54:51,853 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:54:51,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:51,853 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:54:51,853 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-07 08:54:51,853 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-07 08:54:51,853 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:54:51,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:51,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:51,853 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-07 08:54:51,853 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-07 08:54:51,853 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:54:51,854 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-07 08:54:51,854 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:51,854 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-07 08:54:51,855 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-07 08:54:51,855 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:51,855 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:51,855 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-07 08:54:51,855 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-07 08:54:51,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:51,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:54:51,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:54:51,855 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:51,855 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-07 08:54:51,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:54:51,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:54:51,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-07 08:54:51,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:51,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:51,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:51,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:54:51,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-07 08:54:51,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:51,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:51,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:51,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,34559,1691398396467': 2023-08-07 08:54:51,859 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-08-07 08:54:51,859 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-08-07 08:54:51,859 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:54:51,859 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:54:51,859 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-08-07 08:54:51,859 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:54:51,861 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:51,861 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:51,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:51,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:54:51,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:54:51,861 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:51,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:51,861 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:51,861 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:51,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:51,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:51,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:51,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:54:51,861 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:51,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:51,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:51,861 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:51,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:51,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:51,862 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:51,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-07 08:54:51,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:51,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:51,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:51,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:51,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:51,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:54:51,862 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:51,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:51,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:51,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-07 08:54:51,863 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:51,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:51,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:51,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:51,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:54:51,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-07 08:54:51,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:51,864 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:51,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:51,865 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:51,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:51,865 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:51,868 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:51,868 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:51,868 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:51,868 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:51,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:51,868 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:51,868 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:51,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:51,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:51,868 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:51,868 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:51,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:51,868 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-08-07 08:54:51,869 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:54:51,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:51,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:51,868 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:51,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:51,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:51,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:51,868 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:51,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:51,869 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:51,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:51,869 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-08-07 08:54:51,869 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:51,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:51,869 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:51,869 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:54:51,869 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:51,870 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:51,870 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:51,870 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:51,870 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:51,970 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:54:51,970 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:54:51,982 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:54:51,983 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-08-07 08:54:51,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-08-07 08:54:51,984 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-08-07 08:54:51,984 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:51,984 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-08-07 08:54:51,984 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-08-07 08:54:51,984 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:51,984 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:51,985 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:51,985 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:51,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:51,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:51,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:51,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:51,985 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:51,985 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:51,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:51,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:51,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:51,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:51,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:51,986 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:51,986 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:51,986 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:51,986 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:54:51,987 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:51,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:54:51,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:51,987 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-07 08:54:51,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:54:51,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:51,987 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-07 08:54:51,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:54:51,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:51,987 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-07 08:54:51,987 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-07 08:54:51,987 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-07 08:54:51,987 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-07 08:54:51,987 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-07 08:54:51,987 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-07 08:54:51,987 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-07 08:54:51,987 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:51,987 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-07 08:54:51,987 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-07 08:54:51,987 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:51,987 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-07 08:54:51,988 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-07 08:54:51,988 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-07 08:54:51,988 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:51,987 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-07 08:54:51,988 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-07 08:54:51,988 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-07 08:54:51,988 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-07 08:54:51,988 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-07 08:54:51,988 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:54:51,988 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:54:51,988 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f. 2023-08-07 08:54:51,988 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f. started... 2023-08-07 08:54:51,988 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c3321c1fa2ec0c6f32bbed977602bc2f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:54:51,989 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:51,993 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:51,993 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:51,993 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:54:51,994 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:51,994 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:54:52,002 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/.tmp/cf1/1a16a2185fc34cfc83adacc91a5fd438 2023-08-07 08:54:52,019 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/.tmp/cf2/d5151254caf847efb71504b59bcae5f0 2023-08-07 08:54:52,024 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/.tmp/cf1/1a16a2185fc34cfc83adacc91a5fd438 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf1/1a16a2185fc34cfc83adacc91a5fd438 2023-08-07 08:54:52,028 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf1/1a16a2185fc34cfc83adacc91a5fd438, entries=10, sequenceid=19, filesize=5.0 K 2023-08-07 08:54:52,029 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/.tmp/cf2/d5151254caf847efb71504b59bcae5f0 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf2/d5151254caf847efb71504b59bcae5f0 2023-08-07 08:54:52,033 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf2/d5151254caf847efb71504b59bcae5f0, entries=10, sequenceid=19, filesize=5.0 K 2023-08-07 08:54:52,034 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c3321c1fa2ec0c6f32bbed977602bc2f in 46ms, sequenceid=19, compaction requested=false 2023-08-07 08:54:52,034 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c3321c1fa2ec0c6f32bbed977602bc2f: 2023-08-07 08:54:52,034 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f. 2023-08-07 08:54:52,034 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:54:52,034 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:54:52,035 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,035 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-07 08:54:52,035 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-07 08:54:52,036 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,036 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:54:52,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:54:52,036 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,036 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:54:52,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:54:52,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:54:52,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-07 08:54:52,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:54:52,039 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-08-07 08:54:52,039 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@fa6cfc4[Count = 0] remaining members to acquire global barrier 2023-08-07 08:54:52,039 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-08-07 08:54:52,039 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,040 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,040 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,040 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,040 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,040 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,040 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-07 08:54:52,040 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,040 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-07 08:54:52,040 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-07 08:54:52,040 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-07 08:54:52,040 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:54:52,040 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,040 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:54:52,040 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-07 08:54:52,040 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-07 08:54:52,041 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:54:52,041 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:54:52,043 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-07 08:54:52,043 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:52,043 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,043 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-07 08:54:52,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:54:52,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:54:52,044 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-07 08:54:52,044 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:52,044 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-07 08:54:52,044 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-07 08:54:52,044 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:52,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:54:52,044 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-07 08:54:52,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:54:52,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-07 08:54:52,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:54:52,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-07 08:54:52,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,34559,1691398396467': 2023-08-07 08:54:52,048 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-08-07 08:54:52,048 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-08-07 08:54:52,048 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:54:52,048 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:54:52,048 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-08-07 08:54:52,048 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:54:52,049 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,049 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,049 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,049 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:54:52,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:54:52,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,050 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:52,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,049 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:52,049 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:52,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:54:52,050 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:52,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:52,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:52,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:52,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:52,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:52,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-07 08:54:52,051 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:54:52,051 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-07 08:54:52,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:54:52,053 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-07 08:54:52,053 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,053 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,057 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,057 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:52,057 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:52,057 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:52,057 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:52,057 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:52,057 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:52,057 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:52,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:52,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:52,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:52,057 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-08-07 08:54:52,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:52,057 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:52,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:52,057 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,058 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,058 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:52,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:52,057 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:54:52,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:52,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:52,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:52,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:52,058 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,058 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,058 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-08-07 08:54:52,058 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:54:52,058 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,059 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,059 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,059 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,159 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:54:52,160 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:54:52,174 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:54:52,175 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-08-07 08:54:52,175 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-08-07 08:54:52,175 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-08-07 08:54:52,175 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:52,175 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-08-07 08:54:52,175 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-08-07 08:54:52,176 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,176 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,177 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:52,178 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:52,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:52,177 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:52,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:52,178 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:52,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:52,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:52,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:52,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,178 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,178 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:54:52,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,178 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,179 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:54:52,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,179 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-07 08:54:52,179 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,179 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-07 08:54:52,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:54:52,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,179 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-07 08:54:52,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:54:52,179 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-07 08:54:52,179 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-07 08:54:52,179 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:52,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,179 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-07 08:54:52,179 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-07 08:54:52,179 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-07 08:54:52,180 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-07 08:54:52,180 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-07 08:54:52,180 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:54:52,180 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f. 2023-08-07 08:54:52,180 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:54:52,180 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f. started... 2023-08-07 08:54:52,180 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-07 08:54:52,180 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:52,180 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c3321c1fa2ec0c6f32bbed977602bc2f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:54:52,180 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-07 08:54:52,180 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-07 08:54:52,180 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-07 08:54:52,180 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-07 08:54:52,180 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-07 08:54:52,180 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:52,180 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-07 08:54:52,180 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-07 08:54:52,180 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-07 08:54:52,182 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,182 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,182 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,182 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:54:52,183 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,183 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:54:52,193 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/.tmp/cf1/c4e15e83702845e9b2adc0e89362510d 2023-08-07 08:54:52,207 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/.tmp/cf2/43cc827676744606bf19b7a1c7337929 2023-08-07 08:54:52,211 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/.tmp/cf1/c4e15e83702845e9b2adc0e89362510d as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf1/c4e15e83702845e9b2adc0e89362510d 2023-08-07 08:54:52,215 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf1/c4e15e83702845e9b2adc0e89362510d, entries=10, sequenceid=32, filesize=5.0 K 2023-08-07 08:54:52,216 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/.tmp/cf2/43cc827676744606bf19b7a1c7337929 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf2/43cc827676744606bf19b7a1c7337929 2023-08-07 08:54:52,220 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf2/43cc827676744606bf19b7a1c7337929, entries=10, sequenceid=32, filesize=5.0 K 2023-08-07 08:54:52,221 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c3321c1fa2ec0c6f32bbed977602bc2f in 41ms, sequenceid=32, compaction requested=true 2023-08-07 08:54:52,221 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c3321c1fa2ec0c6f32bbed977602bc2f: 2023-08-07 08:54:52,221 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f. 2023-08-07 08:54:52,221 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:54:52,221 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:54:52,221 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,221 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-07 08:54:52,221 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-07 08:54:52,227 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,227 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:54:52,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:54:52,227 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,227 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:54:52,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:54:52,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:54:52,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-07 08:54:52,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:54:52,229 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-08-07 08:54:52,229 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7d380d21[Count = 0] remaining members to acquire global barrier 2023-08-07 08:54:52,229 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-08-07 08:54:52,229 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,230 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,230 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,230 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,231 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-07 08:54:52,231 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,231 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-07 08:54:52,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,231 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:54:52,231 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,231 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:54:52,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,231 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-07 08:54:52,231 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-07 08:54:52,231 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-07 08:54:52,231 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-07 08:54:52,231 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:54:52,231 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:54:52,232 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-07 08:54:52,232 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:52,232 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-07 08:54:52,233 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-07 08:54:52,233 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:52,233 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-07 08:54:52,233 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:52,233 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-07 08:54:52,233 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,233 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-07 08:54:52,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:54:52,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:54:52,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:54:52,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:54:52,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-07 08:54:52,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:54:52,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-07 08:54:52,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,34559,1691398396467': 2023-08-07 08:54:52,237 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-08-07 08:54:52,237 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-08-07 08:54:52,237 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:54:52,237 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:54:52,237 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-08-07 08:54:52,237 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:54:52,240 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,240 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,240 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:52,240 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,240 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,240 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:52,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:54:52,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:54:52,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,240 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:52,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:52,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:54:52,241 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:52,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:52,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:52,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:52,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:52,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,241 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-07 08:54:52,242 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:54:52,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-07 08:54:52,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:54:52,243 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-07 08:54:52,244 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,244 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,248 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:52,248 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:52,248 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:52,248 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:52,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:52,248 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,249 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-08-07 08:54:52,249 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:54:52,248 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:52,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:52,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:52,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:52,248 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:52,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:52,249 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-08-07 08:54:52,249 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,249 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:52,249 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:52,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:52,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:52,249 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,249 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:54:52,250 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:52,250 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:52,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:52,249 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:52,250 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,250 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,250 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,250 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,250 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,250 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,350 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:54:52,351 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:54:52,362 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:54:52,364 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-08-07 08:54:52,364 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-08-07 08:54:52,364 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-08-07 08:54:52,364 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:52,364 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-08-07 08:54:52,364 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-08-07 08:54:52,365 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,365 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,366 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:52,366 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:52,366 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:52,366 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:52,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:52,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:52,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:52,367 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,367 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:54:52,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:52,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:52,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,368 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,368 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,368 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:54:52,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,368 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-07 08:54:52,369 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-07 08:54:52,369 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-07 08:54:52,369 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-07 08:54:52,369 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:52,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:54:52,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:54:52,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,369 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-07 08:54:52,369 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-07 08:54:52,369 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-07 08:54:52,369 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-07 08:54:52,369 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-07 08:54:52,369 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-07 08:54:52,369 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-07 08:54:52,369 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-07 08:54:52,370 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-07 08:54:52,370 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-07 08:54:52,370 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-07 08:54:52,370 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:52,370 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:52,370 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-07 08:54:52,370 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-07 08:54:52,370 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-07 08:54:52,370 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-07 08:54:52,370 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:54:52,370 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:54:52,370 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f. 2023-08-07 08:54:52,370 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f. started... 2023-08-07 08:54:52,370 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c3321c1fa2ec0c6f32bbed977602bc2f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:54:52,371 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,373 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,373 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,373 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:54:52,373 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,373 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:54:52,384 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/.tmp/cf1/7b92d3bc5e5645b6a59dc280d05015ec 2023-08-07 08:54:52,396 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/.tmp/cf2/e33961decc0c480dbddd3fb7850379bd 2023-08-07 08:54:52,400 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/.tmp/cf1/7b92d3bc5e5645b6a59dc280d05015ec as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf1/7b92d3bc5e5645b6a59dc280d05015ec 2023-08-07 08:54:52,404 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf1/7b92d3bc5e5645b6a59dc280d05015ec, entries=10, sequenceid=45, filesize=5.0 K 2023-08-07 08:54:52,404 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/.tmp/cf2/e33961decc0c480dbddd3fb7850379bd as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf2/e33961decc0c480dbddd3fb7850379bd 2023-08-07 08:54:52,408 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf2/e33961decc0c480dbddd3fb7850379bd, entries=10, sequenceid=45, filesize=5.0 K 2023-08-07 08:54:52,409 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c3321c1fa2ec0c6f32bbed977602bc2f in 39ms, sequenceid=45, compaction requested=true 2023-08-07 08:54:52,409 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c3321c1fa2ec0c6f32bbed977602bc2f: 2023-08-07 08:54:52,409 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f. 2023-08-07 08:54:52,409 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:54:52,409 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:54:52,409 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,409 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-07 08:54:52,409 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-07 08:54:52,411 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,411 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:54:52,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:54:52,411 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,411 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:54:52,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:54:52,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:54:52,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-07 08:54:52,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:54:52,413 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-08-07 08:54:52,413 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@34a80a98[Count = 0] remaining members to acquire global barrier 2023-08-07 08:54:52,413 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-08-07 08:54:52,413 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,414 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,414 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,414 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,415 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-07 08:54:52,415 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,415 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-07 08:54:52,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,415 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,415 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:54:52,415 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-07 08:54:52,415 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-07 08:54:52,415 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-07 08:54:52,415 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:54:52,415 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:54:52,415 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-07 08:54:52,415 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:54:52,416 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-07 08:54:52,416 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:52,416 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-07 08:54:52,416 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:52,416 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-07 08:54:52,416 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-07 08:54:52,418 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-07 08:54:52,418 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,418 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:52,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:54:52,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:54:52,418 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-07 08:54:52,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:54:52,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:54:52,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-07 08:54:52,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:54:52,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-07 08:54:52,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,34559,1691398396467': 2023-08-07 08:54:52,422 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-08-07 08:54:52,422 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-08-07 08:54:52,422 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:54:52,422 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:54:52,422 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-08-07 08:54:52,422 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:54:52,423 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,423 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,423 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,423 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:52,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,423 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:52,423 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:54:52,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:54:52,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,424 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:52,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,424 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:52,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:54:52,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:52,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:52,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:52,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:52,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:52,424 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-07 08:54:52,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,425 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:54:52,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-07 08:54:52,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,426 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:54:52,427 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-07 08:54:52,427 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,430 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,430 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:52,430 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:52,430 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:52,430 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:52,430 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:52,430 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:52,430 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:52,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:52,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:52,430 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:52,430 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-08-07 08:54:52,431 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:54:52,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:52,430 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:52,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:52,431 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,431 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-08-07 08:54:52,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:52,431 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:54:52,431 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:52,431 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,431 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,431 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,431 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,431 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,431 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:52,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:52,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:52,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:52,531 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:54:52,532 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:54:52,544 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:54:52,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-08-07 08:54:52,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-08-07 08:54:52,545 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-08-07 08:54:52,545 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:52,545 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-08-07 08:54:52,545 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-08-07 08:54:52,546 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,546 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,547 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,547 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:52,547 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:52,547 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:52,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:52,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:52,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:52,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:52,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:52,547 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,547 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:54:52,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:52,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,548 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,548 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,548 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:54:52,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:54:52,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,548 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-07 08:54:52,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:54:52,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,548 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-07 08:54:52,549 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-07 08:54:52,549 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-07 08:54:52,549 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-07 08:54:52,549 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-07 08:54:52,549 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-07 08:54:52,549 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-07 08:54:52,549 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-07 08:54:52,549 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-07 08:54:52,549 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:52,549 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-07 08:54:52,549 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-07 08:54:52,549 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:52,549 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-07 08:54:52,549 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-07 08:54:52,549 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-07 08:54:52,549 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:54:52,577 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:54:52,549 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:52,577 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-07 08:54:52,577 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-07 08:54:52,577 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-07 08:54:52,577 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f. 2023-08-07 08:54:52,577 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f. started... 2023-08-07 08:54:52,549 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-07 08:54:52,578 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c3321c1fa2ec0c6f32bbed977602bc2f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:54:52,580 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,581 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,581 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,581 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:54:52,581 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,581 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:54:52,587 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/.tmp/cf1/debff2d6943948ddba678e130030c6e0 2023-08-07 08:54:52,597 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/.tmp/cf2/e9a580fde6c54740bb901cd34ce1bf82 2023-08-07 08:54:52,601 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/.tmp/cf1/debff2d6943948ddba678e130030c6e0 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf1/debff2d6943948ddba678e130030c6e0 2023-08-07 08:54:52,605 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf1/debff2d6943948ddba678e130030c6e0, entries=10, sequenceid=58, filesize=5.0 K 2023-08-07 08:54:52,606 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/.tmp/cf2/e9a580fde6c54740bb901cd34ce1bf82 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf2/e9a580fde6c54740bb901cd34ce1bf82 2023-08-07 08:54:52,609 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf2/e9a580fde6c54740bb901cd34ce1bf82, entries=10, sequenceid=58, filesize=5.0 K 2023-08-07 08:54:52,610 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c3321c1fa2ec0c6f32bbed977602bc2f in 32ms, sequenceid=58, compaction requested=true 2023-08-07 08:54:52,610 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c3321c1fa2ec0c6f32bbed977602bc2f: 2023-08-07 08:54:52,610 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f. 2023-08-07 08:54:52,610 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:54:52,610 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:54:52,610 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,610 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-07 08:54:52,610 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-07 08:54:52,612 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,612 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:54:52,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:54:52,612 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,612 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:54:52,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:54:52,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:54:52,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-07 08:54:52,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:54:52,614 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-08-07 08:54:52,614 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@454a8418[Count = 0] remaining members to acquire global barrier 2023-08-07 08:54:52,614 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-08-07 08:54:52,614 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,615 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,615 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,615 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,616 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-07 08:54:52,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,616 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,616 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-07 08:54:52,616 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-07 08:54:52,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,616 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:54:52,616 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:54:52,616 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-07 08:54:52,616 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-07 08:54:52,616 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-07 08:54:52,616 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:54:52,616 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:54:52,620 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-07 08:54:52,620 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:52,621 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-07 08:54:52,623 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-07 08:54:52,623 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,623 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:52,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:54:52,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:54:52,623 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-07 08:54:52,623 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-07 08:54:52,624 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:52,624 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-07 08:54:52,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:54:52,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:54:52,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-07 08:54:52,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:54:52,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-07 08:54:52,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,34559,1691398396467': 2023-08-07 08:54:52,627 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-08-07 08:54:52,627 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-08-07 08:54:52,627 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:54:52,627 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:54:52,627 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-08-07 08:54:52,627 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:54:52,630 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,630 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,630 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,630 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,630 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:52,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,630 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:52,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:54:52,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:54:52,630 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,630 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:52,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:54:52,631 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:52,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:52,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:52,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:52,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:52,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:52,631 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-07 08:54:52,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:54:52,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-07 08:54:52,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,633 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:54:52,633 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-07 08:54:52,634 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,636 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,636 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,636 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,636 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:52,636 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:52,636 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:52,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:52,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:52,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:52,636 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:52,636 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,637 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:52,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:52,637 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:52,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:52,636 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-08-07 08:54:52,637 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:54:52,636 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:52,636 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:52,637 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:52,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:52,637 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,637 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:52,637 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:52,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:52,637 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-08-07 08:54:52,637 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:54:52,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:52,637 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,638 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,638 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,638 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,638 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,638 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,738 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:54:52,739 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:54:52,750 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:54:52,751 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-08-07 08:54:52,752 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-08-07 08:54:52,752 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-08-07 08:54:52,752 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:52,752 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-08-07 08:54:52,752 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-08-07 08:54:52,752 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,752 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,753 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:52,753 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:52,753 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:52,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:52,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:52,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:52,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:52,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:52,753 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:52,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,754 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,754 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:54:52,754 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,754 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,754 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:54:52,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,755 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-07 08:54:52,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:54:52,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:54:52,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,755 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-07 08:54:52,755 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-07 08:54:52,755 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-07 08:54:52,755 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-07 08:54:52,755 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-07 08:54:52,755 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-07 08:54:52,755 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-07 08:54:52,755 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-07 08:54:52,755 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:52,755 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-07 08:54:52,755 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-07 08:54:52,756 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-07 08:54:52,756 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-07 08:54:52,755 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-07 08:54:52,756 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:52,756 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-07 08:54:52,756 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-07 08:54:52,756 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:52,756 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-07 08:54:52,756 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-07 08:54:52,756 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-07 08:54:52,756 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:54:52,756 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:54:52,756 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f. 2023-08-07 08:54:52,756 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f. started... 2023-08-07 08:54:52,756 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c3321c1fa2ec0c6f32bbed977602bc2f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:54:52,757 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,759 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,759 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,759 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:54:52,759 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,759 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:54:52,766 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/.tmp/cf1/df4cdbc6cd234ebb80a9f226e5fef26b 2023-08-07 08:54:52,780 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/.tmp/cf2/fbfb56a7bde54fdc885682e88865142c 2023-08-07 08:54:52,784 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/.tmp/cf1/df4cdbc6cd234ebb80a9f226e5fef26b as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf1/df4cdbc6cd234ebb80a9f226e5fef26b 2023-08-07 08:54:52,788 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf1/df4cdbc6cd234ebb80a9f226e5fef26b, entries=10, sequenceid=71, filesize=5.0 K 2023-08-07 08:54:52,788 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/.tmp/cf2/fbfb56a7bde54fdc885682e88865142c as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf2/fbfb56a7bde54fdc885682e88865142c 2023-08-07 08:54:52,791 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf2/fbfb56a7bde54fdc885682e88865142c, entries=10, sequenceid=71, filesize=5.0 K 2023-08-07 08:54:52,792 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c3321c1fa2ec0c6f32bbed977602bc2f in 36ms, sequenceid=71, compaction requested=true 2023-08-07 08:54:52,792 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c3321c1fa2ec0c6f32bbed977602bc2f: 2023-08-07 08:54:52,792 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f. 2023-08-07 08:54:52,792 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:54:52,792 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:54:52,792 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,792 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-07 08:54:52,792 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-07 08:54:52,794 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,794 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:54:52,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:54:52,794 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,794 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:54:52,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:54:52,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:54:52,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-07 08:54:52,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:54:52,796 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-08-07 08:54:52,796 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7cabab9a[Count = 0] remaining members to acquire global barrier 2023-08-07 08:54:52,796 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-08-07 08:54:52,796 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,797 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,797 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,797 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,797 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,797 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:54:52,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,797 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-07 08:54:52,797 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-07 08:54:52,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,797 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-07 08:54:52,798 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-07 08:54:52,798 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:54:52,798 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-07 08:54:52,797 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:54:52,798 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-07 08:54:52,798 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:54:52,799 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-07 08:54:52,799 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:52,799 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-07 08:54:52,801 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-07 08:54:52,801 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,801 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-07 08:54:52,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:54:52,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:54:52,801 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:52,801 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:52,801 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-07 08:54:52,801 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-07 08:54:52,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:54:52,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:54:52,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-07 08:54:52,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:54:52,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-07 08:54:52,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,34559,1691398396467': 2023-08-07 08:54:52,804 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-08-07 08:54:52,804 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-08-07 08:54:52,804 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:54:52,804 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:54:52,804 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-08-07 08:54:52,804 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:54:52,806 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,806 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,806 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:52,806 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:52,806 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,806 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:54:52,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:54:52,806 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:52,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,806 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:52,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:52,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:54:52,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:52,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:52,807 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:52,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:52,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,807 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-07 08:54:52,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:54:52,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-07 08:54:52,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,809 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,809 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:54:52,809 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-07 08:54:52,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,812 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:52,812 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:52,812 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:52,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:52,812 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:52,812 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:52,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:52,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:52,812 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-08-07 08:54:52,812 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:54:52,812 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:52,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:52,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:52,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:52,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:52,812 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,813 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-08-07 08:54:52,812 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:52,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:52,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:52,812 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:52,813 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:54:52,813 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,813 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,813 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-07 08:54:52,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:52,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:52,813 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:52,813 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,813 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,813 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:52,813 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-07 08:54:52,813 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-07 08:54:52,913 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:54:52,914 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:54:52,921 INFO [Listener at localhost/33561] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,923 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34559] regionserver.StoreUtils(129): cannot split hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf1/1a16a2185fc34cfc83adacc91a5fd438 because midkey is the same as first or last row 2023-08-07 08:54:52,923 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34559] regionserver.StoreUtils(129): cannot split hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf2/d5151254caf847efb71504b59bcae5f0 because midkey is the same as first or last row 2023-08-07 08:54:52,923 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34559] regionserver.HRegion(2745): Flushing c3321c1fa2ec0c6f32bbed977602bc2f 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-08-07 08:54:52,932 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34559] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/.tmp/cf1/1d0eaf1dc75f4a279c9122dbaddb2607 2023-08-07 08:54:52,934 ERROR [BP-1940969354-172.31.14.131-1691398391279 heartbeating to localhost/127.0.0.1:45787] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1940969354-172.31.14.131-1691398391279 (Datanode Uuid 5daa8478-5963-4c33-b92f-e21f18db34d9) service to localhost/127.0.0.1:45787 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-08-07 08:54:52,944 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34559] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/.tmp/cf2/5bc0a487b07541b185785a81a8f0f064 2023-08-07 08:54:52,948 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34559] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/.tmp/cf1/1d0eaf1dc75f4a279c9122dbaddb2607 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf1/1d0eaf1dc75f4a279c9122dbaddb2607 2023-08-07 08:54:52,951 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34559] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf1/1d0eaf1dc75f4a279c9122dbaddb2607, entries=8, sequenceid=82, filesize=5.0 K 2023-08-07 08:54:52,952 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34559] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/.tmp/cf2/5bc0a487b07541b185785a81a8f0f064 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf2/5bc0a487b07541b185785a81a8f0f064 2023-08-07 08:54:52,956 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34559] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf2/5bc0a487b07541b185785a81a8f0f064, entries=8, sequenceid=82, filesize=5.0 K 2023-08-07 08:54:52,957 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34559] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for c3321c1fa2ec0c6f32bbed977602bc2f in 34ms, sequenceid=82, compaction requested=true 2023-08-07 08:54:52,958 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34559] regionserver.HRegion(2446): Flush status journal for c3321c1fa2ec0c6f32bbed977602bc2f: 2023-08-07 08:54:52,958 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34559] regionserver.StoreUtils(129): cannot split hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf1/1a16a2185fc34cfc83adacc91a5fd438 because midkey is the same as first or last row 2023-08-07 08:54:52,958 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34559] regionserver.StoreUtils(129): cannot split hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf2/d5151254caf847efb71504b59bcae5f0 because midkey is the same as first or last row 2023-08-07 08:54:52,958 DEBUG [Listener at localhost/33561] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,959 DEBUG [Listener at localhost/33561] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-08-07 08:54:52,959 WARN [Listener at localhost/33561] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-07 08:54:52,960 INFO [Listener at localhost/33561] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-08-07 08:54:52,961 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testInvalidSplitKey 2023-08-07 08:54:52,962 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:52,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-08-07 08:54:52,964 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398492964"}]},"ts":"1691398492964"} 2023-08-07 08:54:52,965 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-08-07 08:54:52,967 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-08-07 08:54:52,968 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=c3321c1fa2ec0c6f32bbed977602bc2f, UNASSIGN}] 2023-08-07 08:54:52,969 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=c3321c1fa2ec0c6f32bbed977602bc2f, UNASSIGN 2023-08-07 08:54:52,969 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=c3321c1fa2ec0c6f32bbed977602bc2f, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:52,969 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1691398492969"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398492969"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398492969"}]},"ts":"1691398492969"} 2023-08-07 08:54:52,970 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure c3321c1fa2ec0c6f32bbed977602bc2f, server=jenkins-hbase4.apache.org,34559,1691398396467}] 2023-08-07 08:54:53,065 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-08-07 08:54:53,122 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close c3321c1fa2ec0c6f32bbed977602bc2f 2023-08-07 08:54:53,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing c3321c1fa2ec0c6f32bbed977602bc2f, disabling compactions & flushes 2023-08-07 08:54:53,122 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f. 2023-08-07 08:54:53,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f. 2023-08-07 08:54:53,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f. after waiting 0 ms 2023-08-07 08:54:53,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f. 2023-08-07 08:54:53,126 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-08-07 08:54:53,127 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-07 08:54:53,127 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f. 2023-08-07 08:54:53,127 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for c3321c1fa2ec0c6f32bbed977602bc2f: 2023-08-07 08:54:53,128 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed c3321c1fa2ec0c6f32bbed977602bc2f 2023-08-07 08:54:53,129 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=c3321c1fa2ec0c6f32bbed977602bc2f, regionState=CLOSED 2023-08-07 08:54:53,129 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1691398493128"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398493128"}]},"ts":"1691398493128"} 2023-08-07 08:54:53,131 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=106 2023-08-07 08:54:53,131 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure c3321c1fa2ec0c6f32bbed977602bc2f, server=jenkins-hbase4.apache.org,34559,1691398396467 in 160 msec 2023-08-07 08:54:53,132 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-08-07 08:54:53,132 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=c3321c1fa2ec0c6f32bbed977602bc2f, UNASSIGN in 163 msec 2023-08-07 08:54:53,133 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398493133"}]},"ts":"1691398493133"} 2023-08-07 08:54:53,134 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-08-07 08:54:53,135 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-08-07 08:54:53,137 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 175 msec 2023-08-07 08:54:53,266 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-08-07 08:54:53,266 INFO [Listener at localhost/33561] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-08-07 08:54:53,267 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testInvalidSplitKey 2023-08-07 08:54:53,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:53,269 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:53,270 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:53,270 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=108 2023-08-07 08:54:53,273 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f 2023-08-07 08:54:53,275 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf1, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf2, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/recovered.edits] 2023-08-07 08:54:53,278 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf1/1a16a2185fc34cfc83adacc91a5fd438 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf1/1a16a2185fc34cfc83adacc91a5fd438 2023-08-07 08:54:53,279 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf1/1d0eaf1dc75f4a279c9122dbaddb2607 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf1/1d0eaf1dc75f4a279c9122dbaddb2607 2023-08-07 08:54:53,280 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf1/7b92d3bc5e5645b6a59dc280d05015ec to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf1/7b92d3bc5e5645b6a59dc280d05015ec 2023-08-07 08:54:53,282 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf1/c4e15e83702845e9b2adc0e89362510d to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf1/c4e15e83702845e9b2adc0e89362510d 2023-08-07 08:54:53,283 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf1/db4278d19b3b46b99d03e79749307867 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf1/db4278d19b3b46b99d03e79749307867 2023-08-07 08:54:53,284 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf1/debff2d6943948ddba678e130030c6e0 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf1/debff2d6943948ddba678e130030c6e0 2023-08-07 08:54:53,285 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf1/df4cdbc6cd234ebb80a9f226e5fef26b to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf1/df4cdbc6cd234ebb80a9f226e5fef26b 2023-08-07 08:54:53,288 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf2/43cc827676744606bf19b7a1c7337929 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf2/43cc827676744606bf19b7a1c7337929 2023-08-07 08:54:53,289 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf2/44cfd9773a9e422a93d5d0e31ad82f94 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf2/44cfd9773a9e422a93d5d0e31ad82f94 2023-08-07 08:54:53,290 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf2/5bc0a487b07541b185785a81a8f0f064 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf2/5bc0a487b07541b185785a81a8f0f064 2023-08-07 08:54:53,291 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf2/d5151254caf847efb71504b59bcae5f0 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf2/d5151254caf847efb71504b59bcae5f0 2023-08-07 08:54:53,292 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf2/e33961decc0c480dbddd3fb7850379bd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf2/e33961decc0c480dbddd3fb7850379bd 2023-08-07 08:54:53,293 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf2/e9a580fde6c54740bb901cd34ce1bf82 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf2/e9a580fde6c54740bb901cd34ce1bf82 2023-08-07 08:54:53,294 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf2/fbfb56a7bde54fdc885682e88865142c to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/cf2/fbfb56a7bde54fdc885682e88865142c 2023-08-07 08:54:53,297 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/recovered.edits/85.seqid to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f/recovered.edits/85.seqid 2023-08-07 08:54:53,297 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testInvalidSplitKey/c3321c1fa2ec0c6f32bbed977602bc2f 2023-08-07 08:54:53,297 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-08-07 08:54:53,300 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:53,301 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-08-07 08:54:53,306 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-08-07 08:54:53,307 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:53,307 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-08-07 08:54:53,307 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691398493307"}]},"ts":"9223372036854775807"} 2023-08-07 08:54:53,308 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-08-07 08:54:53,308 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => c3321c1fa2ec0c6f32bbed977602bc2f, NAME => 'testInvalidSplitKey,,1691398491285.c3321c1fa2ec0c6f32bbed977602bc2f.', STARTKEY => '', ENDKEY => ''}] 2023-08-07 08:54:53,308 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-08-07 08:54:53,309 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1691398493308"}]},"ts":"9223372036854775807"} 2023-08-07 08:54:53,310 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-08-07 08:54:53,312 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:53,313 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 45 msec 2023-08-07 08:54:53,371 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=108 2023-08-07 08:54:53,371 INFO [Listener at localhost/33561] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-08-07 08:54:53,386 INFO [Listener at localhost/33561] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=367 (was 348) - Thread LEAK? -, OpenFileDescriptor=743 (was 659) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=202 (was 202), ProcessCount=175 (was 175), AvailableMemoryMB=1821 (was 1897) 2023-08-07 08:54:53,401 INFO [Listener at localhost/33561] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=367, OpenFileDescriptor=743, MaxFileDescriptor=60000, SystemLoadAverage=202, ProcessCount=175, AvailableMemoryMB=1820 2023-08-07 08:54:53,401 WARN [Listener at localhost/33561] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-07 08:54:53,403 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-08-07 08:54:53,404 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-08-07 08:54:53,405 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-08-07 08:54:53,407 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-08-07 08:54:53,407 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-07 08:54:53,409 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76 2023-08-07 08:54:53,409 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76 empty. 2023-08-07 08:54:53,409 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76 2023-08-07 08:54:53,410 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-08-07 08:54:53,419 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-08-07 08:54:53,420 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7675): creating {ENCODED => 824d793ff4148484b7237e455699de76, NAME => 'testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp 2023-08-07 08:54:53,427 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:54:53,427 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1604): Closing 824d793ff4148484b7237e455699de76, disabling compactions & flushes 2023-08-07 08:54:53,427 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76. 2023-08-07 08:54:53,427 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76. 2023-08-07 08:54:53,427 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76. after waiting 0 ms 2023-08-07 08:54:53,427 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76. 2023-08-07 08:54:53,427 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76. 2023-08-07 08:54:53,427 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1558): Region close journal for 824d793ff4148484b7237e455699de76: 2023-08-07 08:54:53,429 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-08-07 08:54:53,429 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691398493429"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398493429"}]},"ts":"1691398493429"} 2023-08-07 08:54:53,431 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-07 08:54:53,431 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-07 08:54:53,431 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398493431"}]},"ts":"1691398493431"} 2023-08-07 08:54:53,432 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-08-07 08:54:53,436 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-07 08:54:53,437 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-07 08:54:53,437 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-07 08:54:53,437 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-07 08:54:53,437 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-07 08:54:53,438 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=824d793ff4148484b7237e455699de76, ASSIGN}] 2023-08-07 08:54:53,439 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=824d793ff4148484b7237e455699de76, ASSIGN 2023-08-07 08:54:53,439 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=824d793ff4148484b7237e455699de76, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34559,1691398396467; forceNewPlan=false, retain=false 2023-08-07 08:54:53,589 INFO [jenkins-hbase4:33357] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-07 08:54:53,591 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=824d793ff4148484b7237e455699de76, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:53,591 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691398493591"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398493591"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398493591"}]},"ts":"1691398493591"} 2023-08-07 08:54:53,592 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure 824d793ff4148484b7237e455699de76, server=jenkins-hbase4.apache.org,34559,1691398396467}] 2023-08-07 08:54:53,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76. 2023-08-07 08:54:53,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 824d793ff4148484b7237e455699de76, NAME => 'testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76.', STARTKEY => '', ENDKEY => ''} 2023-08-07 08:54:53,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-07 08:54:53,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 824d793ff4148484b7237e455699de76 2023-08-07 08:54:53,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:54:53,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 824d793ff4148484b7237e455699de76 2023-08-07 08:54:53,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 824d793ff4148484b7237e455699de76 2023-08-07 08:54:53,748 INFO [StoreOpener-824d793ff4148484b7237e455699de76-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 824d793ff4148484b7237e455699de76 2023-08-07 08:54:53,749 DEBUG [StoreOpener-824d793ff4148484b7237e455699de76-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1 2023-08-07 08:54:53,750 DEBUG [StoreOpener-824d793ff4148484b7237e455699de76-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1 2023-08-07 08:54:53,750 INFO [StoreOpener-824d793ff4148484b7237e455699de76-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 824d793ff4148484b7237e455699de76 columnFamilyName cf1 2023-08-07 08:54:53,750 INFO [StoreOpener-824d793ff4148484b7237e455699de76-1] regionserver.HStore(310): Store=824d793ff4148484b7237e455699de76/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:54:53,750 INFO [StoreOpener-824d793ff4148484b7237e455699de76-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 824d793ff4148484b7237e455699de76 2023-08-07 08:54:53,751 DEBUG [StoreOpener-824d793ff4148484b7237e455699de76-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2 2023-08-07 08:54:53,752 DEBUG [StoreOpener-824d793ff4148484b7237e455699de76-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2 2023-08-07 08:54:53,752 INFO [StoreOpener-824d793ff4148484b7237e455699de76-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 824d793ff4148484b7237e455699de76 columnFamilyName cf2 2023-08-07 08:54:53,752 INFO [StoreOpener-824d793ff4148484b7237e455699de76-1] regionserver.HStore(310): Store=824d793ff4148484b7237e455699de76/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:54:53,753 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76 2023-08-07 08:54:53,754 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76 2023-08-07 08:54:53,755 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-07 08:54:53,756 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 824d793ff4148484b7237e455699de76 2023-08-07 08:54:53,757 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 08:54:53,758 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 824d793ff4148484b7237e455699de76; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=58920949, jitterRate=-0.12200944125652313}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-07 08:54:53,758 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 824d793ff4148484b7237e455699de76: 2023-08-07 08:54:53,758 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76., pid=111, masterSystemTime=1691398493744 2023-08-07 08:54:53,760 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76. 2023-08-07 08:54:53,760 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76. 2023-08-07 08:54:53,760 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=824d793ff4148484b7237e455699de76, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:53,760 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691398493760"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691398493760"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691398493760"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398493760"}]},"ts":"1691398493760"} 2023-08-07 08:54:53,763 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-08-07 08:54:53,763 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure 824d793ff4148484b7237e455699de76, server=jenkins-hbase4.apache.org,34559,1691398396467 in 170 msec 2023-08-07 08:54:53,765 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-08-07 08:54:53,765 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=824d793ff4148484b7237e455699de76, ASSIGN in 325 msec 2023-08-07 08:54:53,766 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-07 08:54:53,766 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398493766"}]},"ts":"1691398493766"} 2023-08-07 08:54:53,767 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-08-07 08:54:53,769 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-08-07 08:54:53,770 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 365 msec 2023-08-07 08:54:53,846 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-08-07 08:54:53,913 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:54:53,914 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-08-07 08:54:53,914 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-08-07 08:54:53,914 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:53,914 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-08-07 08:54:53,914 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-08-07 08:54:53,915 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,915 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,916 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:53,916 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:53,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:53,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:53,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:53,916 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:53,916 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:53,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:53,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:53,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:53,916 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:53,916 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:54:53,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,917 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,917 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,917 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:54:53,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:54:53,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:54:53,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,917 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-07 08:54:53,917 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-07 08:54:53,917 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-07 08:54:53,918 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-07 08:54:53,918 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-07 08:54:53,918 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-07 08:54:53,918 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-07 08:54:53,918 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:53,918 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-07 08:54:53,918 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:53,918 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-07 08:54:53,918 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-07 08:54:53,918 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:53,918 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-07 08:54:53,918 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-07 08:54:53,918 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-07 08:54:53,918 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-07 08:54:53,918 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-07 08:54:53,918 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-07 08:54:53,918 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:54:53,918 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:54:53,918 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76. 2023-08-07 08:54:53,918 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76. started... 2023-08-07 08:54:53,920 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 824d793ff4148484b7237e455699de76 2/2 column families, dataSize=120 B heapSize=928 B 2023-08-07 08:54:53,923 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,923 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,923 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,923 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,923 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:54:53,923 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:54:53,930 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/.tmp/cf1/de1cf1c939b34080981ef8a51826a2fc 2023-08-07 08:54:53,941 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/.tmp/cf2/62417b4c0b964993adddf5033e482896 2023-08-07 08:54:53,945 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/.tmp/cf1/de1cf1c939b34080981ef8a51826a2fc as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/de1cf1c939b34080981ef8a51826a2fc 2023-08-07 08:54:53,949 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/de1cf1c939b34080981ef8a51826a2fc, entries=2, sequenceid=6, filesize=4.8 K 2023-08-07 08:54:53,950 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/.tmp/cf2/62417b4c0b964993adddf5033e482896 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/62417b4c0b964993adddf5033e482896 2023-08-07 08:54:53,953 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/62417b4c0b964993adddf5033e482896, entries=2, sequenceid=6, filesize=4.8 K 2023-08-07 08:54:53,953 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 824d793ff4148484b7237e455699de76 in 33ms, sequenceid=6, compaction requested=false 2023-08-07 08:54:53,954 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 824d793ff4148484b7237e455699de76: 2023-08-07 08:54:53,954 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76. 2023-08-07 08:54:53,954 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:54:53,955 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:54:53,955 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:53,955 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-07 08:54:53,955 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-07 08:54:53,956 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:53,956 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:53,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:54:53,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:54:53,956 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:54:53,956 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:54:53,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:54:53,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-07 08:54:53,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:53,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:53,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:53,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:54:53,958 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-08-07 08:54:53,958 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@41c7e02b[Count = 0] remaining members to acquire global barrier 2023-08-07 08:54:53,958 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-08-07 08:54:53,958 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,959 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,959 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,960 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,960 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:53,960 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:54:53,959 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,960 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,960 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-07 08:54:53,960 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-07 08:54:53,960 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-07 08:54:53,960 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:54:53,960 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-07 08:54:53,960 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:54:53,960 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-07 08:54:53,960 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-07 08:54:53,960 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:54:53,961 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-07 08:54:53,961 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:53,961 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-07 08:54:53,961 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-07 08:54:53,961 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:53,961 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-07 08:54:53,963 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-07 08:54:53,963 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:53,963 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:53,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:53,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:54:53,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:54:53,964 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-07 08:54:53,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:54:53,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:54:53,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-07 08:54:53,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:53,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:53,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:53,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:54:53,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-07 08:54:53,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:53,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:53,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:53,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,34559,1691398396467': 2023-08-07 08:54:53,967 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-08-07 08:54:53,967 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-08-07 08:54:53,967 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:54:53,967 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:54:53,967 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-08-07 08:54:53,967 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:54:53,968 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,968 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,969 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:53,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,969 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:53,969 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,968 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:54:53,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:53,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:53,969 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:53,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:54:53,969 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:53,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:53,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:53,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:53,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:54:53,970 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:53,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:53,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-07 08:54:53,970 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:53,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:54:53,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-07 08:54:53,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:53,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:53,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:53,971 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:53,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:54:53,972 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:53,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-07 08:54:53,972 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:53,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:53,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:53,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:53,974 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:53,975 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:53,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:53,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:53,975 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:53,975 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:53,975 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:53,975 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:53,975 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:53,975 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,975 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-08-07 08:54:53,975 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:54:53,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:53,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:53,975 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:53,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:53,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:53,975 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:53,975 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-08-07 08:54:53,975 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:53,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:53,975 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:54:53,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:53,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:53,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:53,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:53,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:53,975 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,976 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:53,976 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,976 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:53,976 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:53,976 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:53,976 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,076 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:54:54,077 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:54:54,088 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:54:54,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-08-07 08:54:54,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-08-07 08:54:54,090 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-08-07 08:54:54,090 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:54,090 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-08-07 08:54:54,090 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-08-07 08:54:54,090 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,090 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,091 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:54,091 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:54,091 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:54,091 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:54,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:54,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:54,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:54,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:54,092 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,092 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:54:54,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:54,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,092 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,092 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,092 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:54:54,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:54:54,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:54:54,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,093 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-07 08:54:54,093 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-07 08:54:54,093 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-07 08:54:54,093 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-07 08:54:54,093 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-07 08:54:54,093 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-07 08:54:54,093 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-07 08:54:54,093 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:54,093 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-07 08:54:54,093 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-07 08:54:54,093 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-07 08:54:54,094 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-07 08:54:54,093 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-07 08:54:54,094 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:54,094 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-07 08:54:54,094 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-07 08:54:54,094 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-07 08:54:54,095 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-07 08:54:54,095 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:54,095 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-07 08:54:54,095 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-07 08:54:54,095 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-07 08:54:54,096 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,097 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,097 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:54:54,097 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,098 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:54:54,098 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:54:54,098 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76. 2023-08-07 08:54:54,098 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,098 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76. started... 2023-08-07 08:54:54,098 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:54:54,098 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 824d793ff4148484b7237e455699de76 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:54:54,106 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/.tmp/cf1/002fdccfeb544d5ebf4e03d1ff495ff9 2023-08-07 08:54:54,116 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/.tmp/cf2/deb22c69ab9048f1a49e20e6ce9257d8 2023-08-07 08:54:54,119 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/.tmp/cf1/002fdccfeb544d5ebf4e03d1ff495ff9 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/002fdccfeb544d5ebf4e03d1ff495ff9 2023-08-07 08:54:54,122 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/002fdccfeb544d5ebf4e03d1ff495ff9, entries=10, sequenceid=19, filesize=5.0 K 2023-08-07 08:54:54,123 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/.tmp/cf2/deb22c69ab9048f1a49e20e6ce9257d8 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/deb22c69ab9048f1a49e20e6ce9257d8 2023-08-07 08:54:54,126 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/deb22c69ab9048f1a49e20e6ce9257d8, entries=10, sequenceid=19, filesize=5.0 K 2023-08-07 08:54:54,127 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 824d793ff4148484b7237e455699de76 in 29ms, sequenceid=19, compaction requested=false 2023-08-07 08:54:54,127 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 824d793ff4148484b7237e455699de76: 2023-08-07 08:54:54,127 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76. 2023-08-07 08:54:54,127 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:54:54,127 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:54:54,127 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,127 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-07 08:54:54,127 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-07 08:54:54,131 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,131 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:54:54,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:54:54,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:54:54,131 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,131 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:54:54,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:54:54,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-07 08:54:54,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:54:54,133 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-08-07 08:54:54,133 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@354fce41[Count = 0] remaining members to acquire global barrier 2023-08-07 08:54:54,133 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-08-07 08:54:54,133 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,135 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,135 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,135 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,135 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,135 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,135 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:54:54,135 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,135 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-07 08:54:54,135 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-07 08:54:54,135 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-07 08:54:54,135 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,135 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:54:54,135 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-07 08:54:54,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,135 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:54:54,135 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-07 08:54:54,135 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-07 08:54:54,135 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:54:54,137 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-07 08:54:54,137 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:54,137 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-07 08:54:54,137 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,137 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-07 08:54:54,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:54:54,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:54:54,137 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-07 08:54:54,137 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:54,138 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:54,138 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-07 08:54:54,138 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-07 08:54:54,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:54:54,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:54:54,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-07 08:54:54,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:54:54,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-07 08:54:54,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,34559,1691398396467': 2023-08-07 08:54:54,140 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-08-07 08:54:54,140 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-08-07 08:54:54,140 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:54:54,140 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:54:54,141 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-08-07 08:54:54,141 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:54:54,142 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,142 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,142 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,142 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:54,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,142 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,142 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:54,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:54:54,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:54:54,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,142 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:54,142 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:54:54,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:54,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:54,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:54,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:54,143 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:54,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-07 08:54:54,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:54,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,143 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:54:54,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-07 08:54:54,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:54:54,145 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-07 08:54:54,145 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,145 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,149 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,149 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:54,149 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:54,149 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:54,149 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:54,149 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,149 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-08-07 08:54:54,149 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:54:54,149 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:54,149 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,149 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:54,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:54,149 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:54,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:54,149 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:54,150 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-08-07 08:54:54,150 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:54,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:54,149 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:54,149 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:54,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:54,150 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,150 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:54:54,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:54,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:54,150 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:54,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:54,150 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,151 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,151 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,151 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,151 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,250 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:54:54,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:54:54,262 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:54:54,264 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-08-07 08:54:54,264 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-08-07 08:54:54,264 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-08-07 08:54:54,264 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:54,264 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-08-07 08:54:54,264 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-08-07 08:54:54,265 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,265 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,266 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,266 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:54,266 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:54,266 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:54,266 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:54,266 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:54,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:54,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:54,266 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,266 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:54:54,266 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:54,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:54,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,267 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,267 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:54:54,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:54:54,267 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,267 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-07 08:54:54,267 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-07 08:54:54,268 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-07 08:54:54,268 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-07 08:54:54,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:54:54,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,268 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-07 08:54:54,268 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-07 08:54:54,268 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-07 08:54:54,268 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-07 08:54:54,268 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:54,268 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-07 08:54:54,268 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-07 08:54:54,268 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-07 08:54:54,268 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-07 08:54:54,268 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-07 08:54:54,268 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:54,268 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-07 08:54:54,268 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-07 08:54:54,268 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-07 08:54:54,268 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:54,268 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-07 08:54:54,268 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-07 08:54:54,268 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-07 08:54:54,268 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:54:54,269 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:54:54,269 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76. 2023-08-07 08:54:54,269 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76. started... 2023-08-07 08:54:54,269 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 824d793ff4148484b7237e455699de76 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:54:54,270 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,270 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,270 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,270 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:54:54,271 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,271 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:54:54,280 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/.tmp/cf1/fefdc6befe6d48f3a37a99f82fbf9b2a 2023-08-07 08:54:54,291 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/.tmp/cf2/9b3879532a474f5e99feae370e7a6833 2023-08-07 08:54:54,295 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/.tmp/cf1/fefdc6befe6d48f3a37a99f82fbf9b2a as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/fefdc6befe6d48f3a37a99f82fbf9b2a 2023-08-07 08:54:54,298 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/fefdc6befe6d48f3a37a99f82fbf9b2a, entries=10, sequenceid=32, filesize=5.0 K 2023-08-07 08:54:54,299 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/.tmp/cf2/9b3879532a474f5e99feae370e7a6833 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/9b3879532a474f5e99feae370e7a6833 2023-08-07 08:54:54,302 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/9b3879532a474f5e99feae370e7a6833, entries=10, sequenceid=32, filesize=5.0 K 2023-08-07 08:54:54,303 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 824d793ff4148484b7237e455699de76 in 34ms, sequenceid=32, compaction requested=true 2023-08-07 08:54:54,303 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 824d793ff4148484b7237e455699de76: 2023-08-07 08:54:54,303 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76. 2023-08-07 08:54:54,303 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:54:54,303 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:54:54,303 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,303 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-07 08:54:54,303 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-07 08:54:54,305 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,305 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:54:54,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:54:54,306 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,306 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:54:54,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:54:54,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:54:54,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-07 08:54:54,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:54:54,307 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-08-07 08:54:54,307 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@24b4f6b4[Count = 0] remaining members to acquire global barrier 2023-08-07 08:54:54,307 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-08-07 08:54:54,308 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,309 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,309 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,309 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,309 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-07 08:54:54,309 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,309 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:54:54,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,309 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-07 08:54:54,309 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-07 08:54:54,309 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-07 08:54:54,309 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:54:54,309 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-07 08:54:54,309 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:54:54,309 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-07 08:54:54,309 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:54:54,310 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-07 08:54:54,311 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:54,311 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-07 08:54:54,311 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-07 08:54:54,311 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,311 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:54,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:54:54,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:54:54,311 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-07 08:54:54,311 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-07 08:54:54,311 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:54,311 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-07 08:54:54,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:54:54,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:54:54,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-07 08:54:54,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:54:54,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-07 08:54:54,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,34559,1691398396467': 2023-08-07 08:54:54,315 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-08-07 08:54:54,315 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-08-07 08:54:54,315 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:54:54,315 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:54:54,315 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-08-07 08:54:54,315 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:54:54,317 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,317 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,318 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,317 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,318 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:54,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:54:54,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:54:54,318 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:54,318 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,318 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:54,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:54,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:54,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:54,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:54,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:54,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:54,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:54:54,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,319 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-07 08:54:54,319 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:54:54,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-07 08:54:54,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,321 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:54:54,321 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-07 08:54:54,321 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,326 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,326 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,326 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:54,326 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:54,326 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,326 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-08-07 08:54:54,326 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:54:54,326 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:54,326 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:54,326 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:54,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:54,326 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:54,326 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:54,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:54,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:54,326 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:54,326 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,327 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,327 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,327 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,327 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,327 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,327 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,327 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-08-07 08:54:54,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:54,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:54,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:54,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:54,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:54,327 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:54:54,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:54,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:54,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:54,428 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:54:54,429 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:54:54,439 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:54:54,440 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-08-07 08:54:54,440 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-08-07 08:54:54,441 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-08-07 08:54:54,441 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:54,441 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-08-07 08:54:54,441 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-08-07 08:54:54,441 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,441 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,442 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:54,442 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:54,442 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:54,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:54,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:54,442 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:54,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:54,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:54,443 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,443 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:54:54,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:54,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,443 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,443 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:54:54,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,444 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,444 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-07 08:54:54,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:54:54,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,444 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-07 08:54:54,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:54:54,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,444 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-07 08:54:54,444 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-07 08:54:54,444 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-07 08:54:54,444 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-07 08:54:54,444 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-07 08:54:54,444 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-07 08:54:54,444 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:54,445 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-07 08:54:54,445 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-07 08:54:54,445 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-07 08:54:54,445 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-07 08:54:54,445 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:54,445 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-07 08:54:54,445 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-07 08:54:54,445 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-07 08:54:54,445 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-07 08:54:54,445 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-07 08:54:54,445 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:54,445 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-07 08:54:54,445 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-07 08:54:54,445 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:54:54,445 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:54:54,445 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76. 2023-08-07 08:54:54,445 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76. started... 2023-08-07 08:54:54,445 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 824d793ff4148484b7237e455699de76 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:54:54,448 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,448 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,448 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,449 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:54:54,449 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,449 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:54:54,461 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/.tmp/cf1/1827182aa65f48799fe839b1686c377a 2023-08-07 08:54:54,477 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/.tmp/cf2/10b188603bf243919435c2772898d8ac 2023-08-07 08:54:54,482 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/.tmp/cf1/1827182aa65f48799fe839b1686c377a as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/1827182aa65f48799fe839b1686c377a 2023-08-07 08:54:54,486 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/1827182aa65f48799fe839b1686c377a, entries=10, sequenceid=45, filesize=5.0 K 2023-08-07 08:54:54,487 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/.tmp/cf2/10b188603bf243919435c2772898d8ac as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/10b188603bf243919435c2772898d8ac 2023-08-07 08:54:54,491 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/10b188603bf243919435c2772898d8ac, entries=10, sequenceid=45, filesize=5.0 K 2023-08-07 08:54:54,492 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 824d793ff4148484b7237e455699de76 in 47ms, sequenceid=45, compaction requested=true 2023-08-07 08:54:54,492 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 824d793ff4148484b7237e455699de76: 2023-08-07 08:54:54,492 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76. 2023-08-07 08:54:54,492 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:54:54,492 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:54:54,492 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,492 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-07 08:54:54,492 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-07 08:54:54,494 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,494 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:54:54,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:54:54,494 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,494 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:54:54,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:54:54,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:54:54,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-07 08:54:54,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:54:54,497 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-08-07 08:54:54,497 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5e27ec3[Count = 0] remaining members to acquire global barrier 2023-08-07 08:54:54,497 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-08-07 08:54:54,497 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,498 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,498 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,498 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,498 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-07 08:54:54,498 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,498 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:54:54,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,498 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-07 08:54:54,499 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-07 08:54:54,499 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-07 08:54:54,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,499 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:54:54,499 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:54:54,499 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-07 08:54:54,499 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-07 08:54:54,499 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:54:54,502 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-07 08:54:54,502 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:54,502 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-07 08:54:54,502 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,502 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-07 08:54:54,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:54:54,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:54:54,502 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:54,502 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-07 08:54:54,502 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-07 08:54:54,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:54:54,503 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:54,503 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-07 08:54:54,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:54:54,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-07 08:54:54,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:54:54,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-07 08:54:54,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,34559,1691398396467': 2023-08-07 08:54:54,506 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-08-07 08:54:54,506 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-08-07 08:54:54,506 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:54:54,506 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:54:54,506 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-08-07 08:54:54,506 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:54:54,508 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,508 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,508 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,508 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:54:54,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:54:54,508 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:54,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,508 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:54,508 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:54,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:54,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:54,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:54,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:54:54,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:54,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:54,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:54,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,509 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-07 08:54:54,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,509 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:54:54,510 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-07 08:54:54,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:54:54,512 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-07 08:54:54,512 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,512 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,515 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:54,515 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:54,515 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:54,515 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:54,515 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,515 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:54,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:54,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:54,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:54,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:54,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:54,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:54,516 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,517 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,517 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,517 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,517 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,517 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,517 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,517 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,517 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,517 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:54,516 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-08-07 08:54:54,517 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:54:54,516 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:54,516 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:54,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:54,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:54,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:54,518 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-08-07 08:54:54,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:54,518 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:54:54,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:54,618 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:54:54,619 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:54:54,629 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:54:54,630 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-08-07 08:54:54,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-08-07 08:54:54,631 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-08-07 08:54:54,631 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:54,631 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-08-07 08:54:54,631 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-08-07 08:54:54,631 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,631 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,633 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,633 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:54,633 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:54,633 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:54,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:54,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:54,634 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:54,634 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:54:54,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:54,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:54,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:54,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,634 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,634 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,635 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:54:54,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,635 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-07 08:54:54,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:54:54,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:54:54,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,635 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-07 08:54:54,635 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-07 08:54:54,635 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-07 08:54:54,635 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-07 08:54:54,635 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-07 08:54:54,635 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:54,635 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-07 08:54:54,635 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-07 08:54:54,635 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-07 08:54:54,635 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-07 08:54:54,635 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-07 08:54:54,635 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-07 08:54:54,636 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:54:54,636 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:54:54,636 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:54,636 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-07 08:54:54,636 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:54,636 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-07 08:54:54,636 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-07 08:54:54,636 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-07 08:54:54,636 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76. 2023-08-07 08:54:54,636 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-07 08:54:54,636 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-07 08:54:54,636 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-07 08:54:54,636 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76. started... 2023-08-07 08:54:54,636 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 824d793ff4148484b7237e455699de76 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:54:54,637 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,638 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,638 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,638 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:54:54,638 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,638 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:54:54,645 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/.tmp/cf1/95295d00bddc425d9a6ecbaf12982e76 2023-08-07 08:54:54,656 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/.tmp/cf2/f029eb62f11946f9b8e35a73edb12a9e 2023-08-07 08:54:54,659 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/.tmp/cf1/95295d00bddc425d9a6ecbaf12982e76 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/95295d00bddc425d9a6ecbaf12982e76 2023-08-07 08:54:54,663 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/95295d00bddc425d9a6ecbaf12982e76, entries=10, sequenceid=58, filesize=5.0 K 2023-08-07 08:54:54,663 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/.tmp/cf2/f029eb62f11946f9b8e35a73edb12a9e as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/f029eb62f11946f9b8e35a73edb12a9e 2023-08-07 08:54:54,667 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/f029eb62f11946f9b8e35a73edb12a9e, entries=10, sequenceid=58, filesize=5.0 K 2023-08-07 08:54:54,667 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 824d793ff4148484b7237e455699de76 in 31ms, sequenceid=58, compaction requested=true 2023-08-07 08:54:54,667 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 824d793ff4148484b7237e455699de76: 2023-08-07 08:54:54,667 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76. 2023-08-07 08:54:54,667 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:54:54,668 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:54:54,668 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,668 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-07 08:54:54,668 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-07 08:54:54,674 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,674 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:54:54,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:54:54,674 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,675 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:54:54,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:54:54,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:54:54,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-07 08:54:54,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:54:54,676 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-08-07 08:54:54,676 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7bda10ec[Count = 0] remaining members to acquire global barrier 2023-08-07 08:54:54,676 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-08-07 08:54:54,676 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,677 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,677 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,677 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,678 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-07 08:54:54,678 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-07 08:54:54,678 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-07 08:54:54,678 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-07 08:54:54,678 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:54:54,678 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,678 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:54:54,678 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,678 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:54:54,678 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-07 08:54:54,678 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-07 08:54:54,678 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:54:54,679 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-07 08:54:54,679 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:54,679 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-07 08:54:54,680 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-07 08:54:54,680 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-07 08:54:54,680 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:54,680 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:54,680 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,680 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-07 08:54:54,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:54:54,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:54:54,680 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-07 08:54:54,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:54:54,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:54:54,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-07 08:54:54,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:54:54,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-07 08:54:54,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,34559,1691398396467': 2023-08-07 08:54:54,683 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-08-07 08:54:54,683 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-08-07 08:54:54,683 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:54:54,683 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:54:54,683 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-08-07 08:54:54,683 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:54:54,686 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,686 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,686 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,686 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:54,686 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:54:54,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:54:54,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,686 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:54,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,686 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:54,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:54:54,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:54,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,686 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:54,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:54,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:54,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-07 08:54:54,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:54,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:54,687 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:54:54,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,687 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-07 08:54:54,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:54:54,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-07 08:54:54,689 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,689 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,690 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,692 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,692 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,692 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:54,692 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:54,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:54,692 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:54,692 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:54,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:54,692 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-08-07 08:54:54,692 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:54:54,692 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:54,692 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:54,692 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,692 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,692 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,692 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,692 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,693 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,693 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,693 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,693 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:54,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:54,692 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:54,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:54,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:54,692 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-08-07 08:54:54,693 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:54:54,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:54,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:54,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:54,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:54,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:54,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:54,793 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:54:54,794 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:54:54,805 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:54:54,806 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-08-07 08:54:54,806 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-08-07 08:54:54,806 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-08-07 08:54:54,806 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:54,806 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-08-07 08:54:54,806 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-08-07 08:54:54,806 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,806 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,808 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,808 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:54,808 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:54,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:54,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:54,808 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:54,808 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:54,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:54,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:54,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:54,808 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:54:54,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,809 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,809 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,809 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:54:54,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,809 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-07 08:54:54,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:54:54,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:54:54,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,809 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-07 08:54:54,809 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-07 08:54:54,809 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-07 08:54:54,809 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-07 08:54:54,809 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-07 08:54:54,809 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-07 08:54:54,809 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:54,810 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-07 08:54:54,810 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-07 08:54:54,810 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-07 08:54:54,810 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-07 08:54:54,810 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-07 08:54:54,810 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:54,810 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:54:54,810 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-07 08:54:54,810 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-07 08:54:54,810 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-07 08:54:54,810 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-07 08:54:54,810 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-07 08:54:54,810 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-07 08:54:54,810 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:54:54,810 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:54:54,810 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76. 2023-08-07 08:54:54,810 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-07 08:54:54,810 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76. started... 2023-08-07 08:54:54,810 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 824d793ff4148484b7237e455699de76 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:54:54,811 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,813 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,813 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:54:54,813 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,813 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,813 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:54:54,819 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/.tmp/cf1/1c949736b8214e9dbfb50301075ce902 2023-08-07 08:54:54,830 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/.tmp/cf2/5c6b91afece54b158ca8c1f81468b5c1 2023-08-07 08:54:54,834 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/.tmp/cf1/1c949736b8214e9dbfb50301075ce902 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/1c949736b8214e9dbfb50301075ce902 2023-08-07 08:54:54,838 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/1c949736b8214e9dbfb50301075ce902, entries=10, sequenceid=71, filesize=5.0 K 2023-08-07 08:54:54,838 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/.tmp/cf2/5c6b91afece54b158ca8c1f81468b5c1 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/5c6b91afece54b158ca8c1f81468b5c1 2023-08-07 08:54:54,842 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/5c6b91afece54b158ca8c1f81468b5c1, entries=10, sequenceid=71, filesize=5.0 K 2023-08-07 08:54:54,843 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 824d793ff4148484b7237e455699de76 in 32ms, sequenceid=71, compaction requested=true 2023-08-07 08:54:54,843 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 824d793ff4148484b7237e455699de76: 2023-08-07 08:54:54,843 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76. 2023-08-07 08:54:54,843 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:54:54,843 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:54:54,843 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,843 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-07 08:54:54,843 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-07 08:54:54,844 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:54:54,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:54:54,844 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:54:54,845 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,845 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:54:54,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:54:54,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-07 08:54:54,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:54:54,846 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-08-07 08:54:54,846 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@302ce410[Count = 0] remaining members to acquire global barrier 2023-08-07 08:54:54,846 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-08-07 08:54:54,847 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,848 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,848 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,848 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,848 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:54:54,848 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,848 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-07 08:54:54,848 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-07 08:54:54,848 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-07 08:54:54,848 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-07 08:54:54,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,848 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:54:54,848 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:54:54,848 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-07 08:54:54,848 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-07 08:54:54,848 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:54:54,850 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-07 08:54:54,850 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:54,850 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,850 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-07 08:54:54,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:54:54,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:54:54,850 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-07 08:54:54,850 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:54,850 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-07 08:54:54,857 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-07 08:54:54,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:54:54,857 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:54,857 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-07 08:54:54,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:54:54,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-07 08:54:54,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:54:54,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-07 08:54:54,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,34559,1691398396467': 2023-08-07 08:54:54,860 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-08-07 08:54:54,860 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-08-07 08:54:54,860 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:54:54,860 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:54:54,860 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-08-07 08:54:54,860 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:54:54,861 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,861 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,861 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:54:54,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:54:54,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,861 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:54,861 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,862 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:54,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,861 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:54,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,862 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:54,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:54:54,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:54,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:54,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:54,862 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-07 08:54:54,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:54,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:54,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,863 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:54:54,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-07 08:54:54,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,864 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:54:54,865 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-07 08:54:54,865 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,869 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,870 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,870 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:54,870 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:54,870 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:54,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:54,870 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:54:54,870 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:54,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:54,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:54,870 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,870 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:54:54,870 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:54:54,870 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-08-07 08:54:54,870 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:54:54,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:54,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:54,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:54,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:54:54,870 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,870 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:54:54,870 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:54:54,871 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-08-07 08:54:54,871 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:54:54,871 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:54,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:54,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:54:54,871 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:54,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:54:54,871 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:54:54,871 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,871 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-07 08:54:54,971 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:54:54,972 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:54:55,230 WARN [Listener at localhost/33561] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-08-07 08:54:55,231 DEBUG [Listener at localhost/33561] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:55,231 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:55,233 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:55,233 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:55,233 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:55,233 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:54:55,233 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:55,233 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:54:55,233 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:54:55,482 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:55,482 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:54:55,482 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:54:55,482 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:54:55,482 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-08-07 08:54:55,483 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:54:55,483 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:54:55,483 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:54:55,483 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:54:55,484 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:54:55,486 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-08-07 08:54:55,486 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:54:55,486 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:54:55,487 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:54:55,487 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:54:55,487 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:54:55,487 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:54:55,488 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:54:55,488 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:54:55,488 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:54:55,488 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:54:55,489 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-07 08:54:55,489 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-07 08:54:55,489 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:54:55,489 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:54:55,490 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN 2023-08-07 08:54:55,490 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:54:55,490 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:54:55,490 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:54:55,490 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=fd6d700fb2bf2b02eb0b469b2b0d401b, daughterA=9ca78a73cef89f541dcf91876dbfafd9, daughterB=5b3565e5e309ccbd325fa8dc34e561c4 2023-08-07 08:54:55,491 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:54:55,491 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:54:55,491 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:54:55,491 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0d0fd872f4eb265c927f025c1f8dd4da, daughterA=890310475de9dc263dd17491ca380a4c, daughterB=246cb6e862f1133bf06f7e32263c91b4 2023-08-07 08:54:55,492 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:54:55,492 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:54:55,492 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:55,492 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:55,493 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,34559,1691398396467, table=testRollbackForSplitTableRegionWithReplica, region=b0ddf60071e752aa19c5e065b70504c9 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=022268c9253618609543a9a81eddac57, daughterA=3088038bd36f4f725bfab047b7dd3b57, daughterB=60ab8c2a890a3fc5d5fda3e377b4c31f 2023-08-07 08:54:55,493 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:55,493 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:55,493 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:55,494 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:55,494 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:55,494 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-08-07 08:54:55,494 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:55,495 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:54:55,495 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:55,495 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:55,495 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:55,495 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-08-07 08:54:55,496 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:54:55,496 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:54:55,497 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398525497 2023-08-07 08:54:55,498 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:55,498 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:54:55,500 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:54:55,501 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=824d793ff4148484b7237e455699de76, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34559,1691398396467, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467, openSeqNum=2 2023-08-07 08:54:55,501 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:54:55,501 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398555501 2023-08-07 08:54:55,501 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398615501 2023-08-07 08:54:55,501 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-08-07 08:54:55,501 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:54:55,503 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:54:55,503 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:55,504 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:54:55,504 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:54:55,751 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:55,751 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:54:55,751 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:54:55,751 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:54:55,752 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 08:54:55,752 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:54:55,752 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:54:55,752 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:54:55,752 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:54:55,753 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:54:55,754 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-08-07 08:54:55,754 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:54:55,755 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:54:55,755 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:54:55,755 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:54:55,755 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:54:55,756 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:54:55,756 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:54:55,756 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:54:55,756 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:54:55,756 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:54:55,756 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-07 08:54:55,756 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-07 08:54:55,757 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:54:55,757 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:54:55,757 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN 2023-08-07 08:54:55,757 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:54:55,757 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:54:55,757 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:54:55,758 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=fd6d700fb2bf2b02eb0b469b2b0d401b, daughterA=9ca78a73cef89f541dcf91876dbfafd9, daughterB=5b3565e5e309ccbd325fa8dc34e561c4 2023-08-07 08:54:55,758 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:54:55,758 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:54:55,758 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:54:55,758 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0d0fd872f4eb265c927f025c1f8dd4da, daughterA=890310475de9dc263dd17491ca380a4c, daughterB=246cb6e862f1133bf06f7e32263c91b4 2023-08-07 08:54:55,759 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:54:55,759 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:54:55,759 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:55,759 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:55,760 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,34559,1691398396467, table=testRollbackForSplitTableRegionWithReplica, region=b0ddf60071e752aa19c5e065b70504c9 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=022268c9253618609543a9a81eddac57, daughterA=3088038bd36f4f725bfab047b7dd3b57, daughterB=60ab8c2a890a3fc5d5fda3e377b4c31f 2023-08-07 08:54:55,760 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:55,760 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:55,760 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:55,760 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:55,760 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:55,760 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-08-07 08:54:55,760 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:55,761 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:54:55,761 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:55,761 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:55,761 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:55,761 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-08-07 08:54:55,762 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:54:55,762 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:54:55,762 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398525762 2023-08-07 08:54:55,764 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:54:55,764 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:55,766 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:54:55,766 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=824d793ff4148484b7237e455699de76, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34559,1691398396467, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467, openSeqNum=2 2023-08-07 08:54:55,766 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:54:55,766 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398555766 2023-08-07 08:54:55,767 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398615767 2023-08-07 08:54:55,767 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-08-07 08:54:55,767 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:54:55,769 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=824d793ff4148484b7237e455699de76, UNASSIGN}] 2023-08-07 08:54:55,769 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:54:55,769 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:55,769 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:54:55,769 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:54:55,847 ERROR [BP-1940969354-172.31.14.131-1691398391279 heartbeating to localhost/127.0.0.1:45787] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1940969354-172.31.14.131-1691398391279 (Datanode Uuid 4fd49325-1f52-494b-bfd8-92875de86a16) service to localhost/127.0.0.1:45787 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-08-07 08:54:56,017 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:56,017 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:54:56,017 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:54:56,017 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:54:56,017 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 08:54:56,017 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:54:56,017 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:54:56,017 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:54:56,018 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:54:56,018 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:54:56,020 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-08-07 08:54:56,020 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:54:56,020 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:54:56,021 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:54:56,021 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:54:56,021 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:54:56,021 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:54:56,021 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:54:56,021 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:54:56,021 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:54:56,022 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:54:56,022 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-07 08:54:56,022 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-07 08:54:56,022 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:54:56,023 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:54:56,023 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN 2023-08-07 08:54:56,023 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:54:56,023 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:54:56,024 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:54:56,024 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=fd6d700fb2bf2b02eb0b469b2b0d401b, daughterA=9ca78a73cef89f541dcf91876dbfafd9, daughterB=5b3565e5e309ccbd325fa8dc34e561c4 2023-08-07 08:54:56,024 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:54:56,024 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:54:56,024 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:54:56,025 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0d0fd872f4eb265c927f025c1f8dd4da, daughterA=890310475de9dc263dd17491ca380a4c, daughterB=246cb6e862f1133bf06f7e32263c91b4 2023-08-07 08:54:56,025 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:54:56,025 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:54:56,025 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:56,026 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:56,026 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,34559,1691398396467, table=testRollbackForSplitTableRegionWithReplica, region=b0ddf60071e752aa19c5e065b70504c9 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=022268c9253618609543a9a81eddac57, daughterA=3088038bd36f4f725bfab047b7dd3b57, daughterB=60ab8c2a890a3fc5d5fda3e377b4c31f 2023-08-07 08:54:56,026 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:56,026 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:56,027 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:56,027 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:56,027 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:56,027 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-08-07 08:54:56,027 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:56,028 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:54:56,028 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:56,028 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:56,028 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:56,028 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-08-07 08:54:56,028 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:54:56,029 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:54:56,029 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398526029 2023-08-07 08:54:56,029 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:54:56,031 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:56,033 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:54:56,033 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=824d793ff4148484b7237e455699de76, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34559,1691398396467, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467, openSeqNum=2 2023-08-07 08:54:56,033 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:54:56,033 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398556033 2023-08-07 08:54:56,033 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398616033 2023-08-07 08:54:56,033 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-08-07 08:54:56,034 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=824d793ff4148484b7237e455699de76, UNASSIGN}] 2023-08-07 08:54:56,034 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:54:56,036 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=824d793ff4148484b7237e455699de76, UNASSIGN 2023-08-07 08:54:56,036 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=824d793ff4148484b7237e455699de76, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:56,037 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691398496036"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398496036"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398496036"}]},"ts":"1691398496036"} 2023-08-07 08:54:56,038 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure 824d793ff4148484b7237e455699de76, server=jenkins-hbase4.apache.org,34559,1691398396467}] 2023-08-07 08:54:56,190 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 824d793ff4148484b7237e455699de76 2023-08-07 08:54:56,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 824d793ff4148484b7237e455699de76, disabling compactions & flushes 2023-08-07 08:54:56,190 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76. 2023-08-07 08:54:56,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76. 2023-08-07 08:54:56,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76. after waiting 0 ms 2023-08-07 08:54:56,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76. 2023-08-07 08:54:56,190 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 824d793ff4148484b7237e455699de76 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-08-07 08:54:56,198 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/.tmp/cf1/69d045a353d543a08852dc2740e7a53a 2023-08-07 08:54:56,209 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/.tmp/cf2/8a3c47f4f5444f1b871568e51c61c322 2023-08-07 08:54:56,213 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/.tmp/cf1/69d045a353d543a08852dc2740e7a53a as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/69d045a353d543a08852dc2740e7a53a 2023-08-07 08:54:56,216 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/69d045a353d543a08852dc2740e7a53a, entries=8, sequenceid=82, filesize=5.0 K 2023-08-07 08:54:56,217 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/.tmp/cf2/8a3c47f4f5444f1b871568e51c61c322 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/8a3c47f4f5444f1b871568e51c61c322 2023-08-07 08:54:56,220 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/8a3c47f4f5444f1b871568e51c61c322, entries=8, sequenceid=82, filesize=5.0 K 2023-08-07 08:54:56,221 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 824d793ff4148484b7237e455699de76 in 31ms, sequenceid=82, compaction requested=true 2023-08-07 08:54:56,224 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-08-07 08:54:56,225 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-07 08:54:56,226 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76. 2023-08-07 08:54:56,226 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 824d793ff4148484b7237e455699de76: 2023-08-07 08:54:56,227 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 824d793ff4148484b7237e455699de76 2023-08-07 08:54:56,228 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=824d793ff4148484b7237e455699de76, regionState=CLOSED 2023-08-07 08:54:56,228 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691398496228"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398496228"}]},"ts":"1691398496228"} 2023-08-07 08:54:56,230 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-08-07 08:54:56,230 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 824d793ff4148484b7237e455699de76, server=jenkins-hbase4.apache.org,34559,1691398396467 in 191 msec 2023-08-07 08:54:56,232 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-08-07 08:54:56,232 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=824d793ff4148484b7237e455699de76, UNASSIGN in 196 msec 2023-08-07 08:54:56,234 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:54:56,234 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:56,234 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:54:56,234 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:54:56,284 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:56,284 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:54:56,284 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:54:56,284 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:54:56,284 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 08:54:56,284 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:54:56,284 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:54:56,284 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:54:56,284 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:54:56,285 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:54:56,286 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-08-07 08:54:56,286 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-07 08:54:56,286 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-07 08:54:56,286 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:54:56,287 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:54:56,287 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:54:56,287 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:54:56,287 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:54:56,287 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:54:56,287 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:54:56,288 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:54:56,288 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:54:56,288 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:54:56,288 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-07 08:54:56,288 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-07 08:54:56,288 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:54:56,289 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:54:56,289 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN 2023-08-07 08:54:56,289 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:54:56,289 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:54:56,289 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:54:56,289 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=fd6d700fb2bf2b02eb0b469b2b0d401b, daughterA=9ca78a73cef89f541dcf91876dbfafd9, daughterB=5b3565e5e309ccbd325fa8dc34e561c4 2023-08-07 08:54:56,289 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:54:56,289 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:54:56,290 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:54:56,290 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0d0fd872f4eb265c927f025c1f8dd4da, daughterA=890310475de9dc263dd17491ca380a4c, daughterB=246cb6e862f1133bf06f7e32263c91b4 2023-08-07 08:54:56,290 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:54:56,290 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:54:56,290 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:56,290 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:56,290 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,34559,1691398396467, table=testRollbackForSplitTableRegionWithReplica, region=b0ddf60071e752aa19c5e065b70504c9 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=022268c9253618609543a9a81eddac57, daughterA=3088038bd36f4f725bfab047b7dd3b57, daughterB=60ab8c2a890a3fc5d5fda3e377b4c31f 2023-08-07 08:54:56,291 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:56,291 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:56,291 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:56,291 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:56,291 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:56,291 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-08-07 08:54:56,291 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:56,292 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=824d793ff4148484b7237e455699de76, UNASSIGN 2023-08-07 08:54:56,292 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 824d793ff4148484b7237e455699de76, server=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:56,292 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:54:56,292 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:56,292 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:56,292 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:56,292 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=824d793ff4148484b7237e455699de76, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:54:56,292 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 824d793ff4148484b7237e455699de76, server=jenkins-hbase4.apache.org,34559,1691398396467 didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:54:56,292 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-08-07 08:54:56,293 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:54:56,293 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:54:56,294 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398526294 2023-08-07 08:54:56,296 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:56,296 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:54:56,298 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:54:56,298 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=824d793ff4148484b7237e455699de76, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34559,1691398396467, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467, openSeqNum=2 2023-08-07 08:54:56,298 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:54:56,298 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398556298 2023-08-07 08:54:56,298 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398616298 2023-08-07 08:54:56,299 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-07 08:54:56,301 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:54:56,310 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=824d793ff4148484b7237e455699de76, threads=14 2023-08-07 08:54:56,311 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/10b188603bf243919435c2772898d8ac for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:56,311 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/5c6b91afece54b158ca8c1f81468b5c1 for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:56,311 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/62417b4c0b964993adddf5033e482896 for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:56,311 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/8a3c47f4f5444f1b871568e51c61c322 for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:56,311 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/9b3879532a474f5e99feae370e7a6833 for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:56,311 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/deb22c69ab9048f1a49e20e6ce9257d8 for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:56,311 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/f029eb62f11946f9b8e35a73edb12a9e for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:56,311 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/002fdccfeb544d5ebf4e03d1ff495ff9 for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:56,311 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/1827182aa65f48799fe839b1686c377a for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:56,312 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/1c949736b8214e9dbfb50301075ce902 for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:56,313 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/69d045a353d543a08852dc2740e7a53a for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:56,313 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/95295d00bddc425d9a6ecbaf12982e76 for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:56,313 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/de1cf1c939b34080981ef8a51826a2fc for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:56,313 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/fefdc6befe6d48f3a37a99f82fbf9b2a for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:56,455 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/62417b4c0b964993adddf5033e482896 for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:56,470 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/f029eb62f11946f9b8e35a73edb12a9e for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:56,470 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/002fdccfeb544d5ebf4e03d1ff495ff9 for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:56,471 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/8a3c47f4f5444f1b871568e51c61c322 for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:56,499 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/fefdc6befe6d48f3a37a99f82fbf9b2a for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:56,499 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/de1cf1c939b34080981ef8a51826a2fc for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:56,499 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/69d045a353d543a08852dc2740e7a53a for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:56,501 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/95295d00bddc425d9a6ecbaf12982e76 for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:56,503 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/5c6b91afece54b158ca8c1f81468b5c1 for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:56,508 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/1c949736b8214e9dbfb50301075ce902 for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:56,556 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-08-07 08:54:56,826 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/deb22c69ab9048f1a49e20e6ce9257d8 for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:56,891 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/1827182aa65f48799fe839b1686c377a for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:56,891 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/10b188603bf243919435c2772898d8ac for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:57,257 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/9b3879532a474f5e99feae370e7a6833 for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:57,257 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 824d793ff4148484b7237e455699de76 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-08-07 08:54:57,286 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:54:57,286 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:57,286 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:54:57,286 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:54:57,299 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:57,299 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:54:57,299 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:54:57,299 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:54:57,299 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 08:54:57,299 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:54:57,300 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:54:57,300 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:54:57,300 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:54:57,300 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:54:57,301 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-08-07 08:54:57,301 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-07 08:54:57,302 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-07 08:54:57,302 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:54:57,302 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:54:57,302 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:54:57,303 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:54:57,303 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:54:57,303 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:54:57,303 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:54:57,303 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:54:57,304 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:54:57,304 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:54:57,304 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-07 08:54:57,304 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-07 08:54:57,304 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:54:57,305 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:54:57,305 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN 2023-08-07 08:54:57,305 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:54:57,305 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:54:57,305 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:54:57,305 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=fd6d700fb2bf2b02eb0b469b2b0d401b, daughterA=9ca78a73cef89f541dcf91876dbfafd9, daughterB=5b3565e5e309ccbd325fa8dc34e561c4 2023-08-07 08:54:57,306 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:54:57,306 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:54:57,306 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:54:57,306 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0d0fd872f4eb265c927f025c1f8dd4da, daughterA=890310475de9dc263dd17491ca380a4c, daughterB=246cb6e862f1133bf06f7e32263c91b4 2023-08-07 08:54:57,306 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:54:57,306 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:54:57,306 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:57,307 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:57,307 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,34559,1691398396467, table=testRollbackForSplitTableRegionWithReplica, region=b0ddf60071e752aa19c5e065b70504c9 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=022268c9253618609543a9a81eddac57, daughterA=3088038bd36f4f725bfab047b7dd3b57, daughterB=60ab8c2a890a3fc5d5fda3e377b4c31f 2023-08-07 08:54:57,307 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:57,307 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:57,307 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:57,307 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:57,307 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:57,308 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-08-07 08:54:57,308 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:57,308 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=824d793ff4148484b7237e455699de76, UNASSIGN 2023-08-07 08:54:57,308 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 824d793ff4148484b7237e455699de76, server=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:57,308 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:54:57,308 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:57,308 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:57,308 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:57,308 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=824d793ff4148484b7237e455699de76, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:54:57,308 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 824d793ff4148484b7237e455699de76, server=jenkins-hbase4.apache.org,34559,1691398396467 didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:54:57,308 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-08-07 08:54:57,309 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:54:57,310 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:54:57,315 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398527315 2023-08-07 08:54:57,316 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:54:57,317 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:57,319 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:54:57,320 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=824d793ff4148484b7237e455699de76, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34559,1691398396467, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467, openSeqNum=2 2023-08-07 08:54:57,320 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:54:57,320 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398557320 2023-08-07 08:54:57,320 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398617320 2023-08-07 08:54:57,320 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-07 08:54:57,328 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=824d793ff4148484b7237e455699de76, threads=14 2023-08-07 08:54:57,328 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/10b188603bf243919435c2772898d8ac for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:57,328 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/5c6b91afece54b158ca8c1f81468b5c1 for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:57,329 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/62417b4c0b964993adddf5033e482896 for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:57,329 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/8a3c47f4f5444f1b871568e51c61c322 for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:57,329 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/9b3879532a474f5e99feae370e7a6833 for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:57,329 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/f029eb62f11946f9b8e35a73edb12a9e for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:57,329 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/deb22c69ab9048f1a49e20e6ce9257d8 for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:57,329 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/002fdccfeb544d5ebf4e03d1ff495ff9 for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:57,329 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/1827182aa65f48799fe839b1686c377a for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:57,329 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/1c949736b8214e9dbfb50301075ce902 for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:57,330 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/69d045a353d543a08852dc2740e7a53a for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:57,330 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/95295d00bddc425d9a6ecbaf12982e76 for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:57,330 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/de1cf1c939b34080981ef8a51826a2fc for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:57,330 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/fefdc6befe6d48f3a37a99f82fbf9b2a for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:57,470 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/002fdccfeb544d5ebf4e03d1ff495ff9 for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:57,475 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/deb22c69ab9048f1a49e20e6ce9257d8 for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:57,487 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/10b188603bf243919435c2772898d8ac for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:57,497 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/69d045a353d543a08852dc2740e7a53a for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:57,503 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/f029eb62f11946f9b8e35a73edb12a9e for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:57,506 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/fefdc6befe6d48f3a37a99f82fbf9b2a for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:57,506 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/de1cf1c939b34080981ef8a51826a2fc for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:57,512 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/95295d00bddc425d9a6ecbaf12982e76 for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:57,512 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/9b3879532a474f5e99feae370e7a6833 for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:57,513 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/1827182aa65f48799fe839b1686c377a for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:57,899 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/8a3c47f4f5444f1b871568e51c61c322 for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:57,905 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/62417b4c0b964993adddf5033e482896 for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:57,906 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/5c6b91afece54b158ca8c1f81468b5c1 for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:57,935 ERROR [BP-1940969354-172.31.14.131-1691398391279 heartbeating to localhost/127.0.0.1:45787] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1940969354-172.31.14.131-1691398391279 (Datanode Uuid 5daa8478-5963-4c33-b92f-e21f18db34d9) service to localhost/127.0.0.1:45787 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-08-07 08:54:58,186 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/1c949736b8214e9dbfb50301075ce902 for region: 824d793ff4148484b7237e455699de76 2023-08-07 08:54:58,186 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 824d793ff4148484b7237e455699de76 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-08-07 08:54:58,218 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:54:58,223 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-08-07 08:54:58,224 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-08-07 08:54:58,224 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:54:58,224 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:58,224 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:54:58,224 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:54:58,320 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:58,320 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:54:58,321 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:54:58,321 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:54:58,321 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 08:54:58,321 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:54:58,321 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:54:58,321 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:54:58,321 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:54:58,322 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:54:58,323 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-08-07 08:54:58,323 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-07 08:54:58,323 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-07 08:54:58,323 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:54:58,323 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:54:58,324 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:54:58,324 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:54:58,324 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:54:58,324 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:54:58,324 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:54:58,324 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:54:58,324 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:54:58,325 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:54:58,325 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-07 08:54:58,325 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-07 08:54:58,325 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:54:58,325 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:54:58,325 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN 2023-08-07 08:54:58,325 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:54:58,325 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:54:58,326 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:54:58,326 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=fd6d700fb2bf2b02eb0b469b2b0d401b, daughterA=9ca78a73cef89f541dcf91876dbfafd9, daughterB=5b3565e5e309ccbd325fa8dc34e561c4 2023-08-07 08:54:58,326 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:54:58,326 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:54:58,326 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:54:58,326 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0d0fd872f4eb265c927f025c1f8dd4da, daughterA=890310475de9dc263dd17491ca380a4c, daughterB=246cb6e862f1133bf06f7e32263c91b4 2023-08-07 08:54:58,326 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:54:58,326 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:54:58,326 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:58,327 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:58,327 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,34559,1691398396467, table=testRollbackForSplitTableRegionWithReplica, region=b0ddf60071e752aa19c5e065b70504c9 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=022268c9253618609543a9a81eddac57, daughterA=3088038bd36f4f725bfab047b7dd3b57, daughterB=60ab8c2a890a3fc5d5fda3e377b4c31f 2023-08-07 08:54:58,327 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:58,327 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:58,327 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:58,327 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:58,327 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:58,327 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-08-07 08:54:58,328 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:58,328 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=824d793ff4148484b7237e455699de76, UNASSIGN 2023-08-07 08:54:58,328 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 824d793ff4148484b7237e455699de76, server=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:58,328 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:54:58,328 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:58,328 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:58,328 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:58,328 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=824d793ff4148484b7237e455699de76, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:54:58,328 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 824d793ff4148484b7237e455699de76, server=jenkins-hbase4.apache.org,34559,1691398396467 didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:54:58,328 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-08-07 08:54:58,329 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:54:58,330 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:54:58,330 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398528330 2023-08-07 08:54:58,331 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:54:58,332 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:58,334 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:54:58,335 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=824d793ff4148484b7237e455699de76, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34559,1691398396467, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467, openSeqNum=2 2023-08-07 08:54:58,335 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:54:58,335 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398558335 2023-08-07 08:54:58,335 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398618335 2023-08-07 08:54:58,335 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-07 08:54:58,337 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:54:58,339 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:54:58,339 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:58,339 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:54:58,339 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:54:58,585 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:58,585 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:54:58,585 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:54:58,585 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:54:58,585 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 08:54:58,586 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:54:58,586 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:54:58,586 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:54:58,586 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:54:58,586 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:54:58,587 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-08-07 08:54:58,588 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-07 08:54:58,588 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-07 08:54:58,588 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:54:58,588 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:54:58,588 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:54:58,588 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:54:58,588 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:54:58,589 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:54:58,589 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:54:58,589 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:54:58,589 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:54:58,589 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:54:58,589 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-07 08:54:58,589 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-07 08:54:58,589 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:54:58,590 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:54:58,590 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN 2023-08-07 08:54:58,590 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:54:58,590 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:54:58,590 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:54:58,590 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=fd6d700fb2bf2b02eb0b469b2b0d401b, daughterA=9ca78a73cef89f541dcf91876dbfafd9, daughterB=5b3565e5e309ccbd325fa8dc34e561c4 2023-08-07 08:54:58,590 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:54:58,590 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:54:58,591 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:54:58,591 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0d0fd872f4eb265c927f025c1f8dd4da, daughterA=890310475de9dc263dd17491ca380a4c, daughterB=246cb6e862f1133bf06f7e32263c91b4 2023-08-07 08:54:58,591 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:54:58,591 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:54:58,591 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:58,591 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:58,591 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,34559,1691398396467, table=testRollbackForSplitTableRegionWithReplica, region=b0ddf60071e752aa19c5e065b70504c9 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=022268c9253618609543a9a81eddac57, daughterA=3088038bd36f4f725bfab047b7dd3b57, daughterB=60ab8c2a890a3fc5d5fda3e377b4c31f 2023-08-07 08:54:58,591 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:58,592 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:58,592 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:58,592 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:58,592 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:58,592 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-08-07 08:54:58,592 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:58,592 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=824d793ff4148484b7237e455699de76, UNASSIGN 2023-08-07 08:54:58,592 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 824d793ff4148484b7237e455699de76, server=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:58,593 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:54:58,593 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:58,593 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:58,593 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:58,593 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=824d793ff4148484b7237e455699de76, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:54:58,593 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 824d793ff4148484b7237e455699de76, server=jenkins-hbase4.apache.org,34559,1691398396467 didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:54:58,593 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-08-07 08:54:58,594 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:54:58,594 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:54:58,594 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398528594 2023-08-07 08:54:58,595 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:54:58,596 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:58,598 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:54:58,598 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=824d793ff4148484b7237e455699de76, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34559,1691398396467, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467, openSeqNum=2 2023-08-07 08:54:58,598 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:54:58,598 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398558598 2023-08-07 08:54:58,598 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398618598 2023-08-07 08:54:58,598 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-07 08:54:58,599 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:54:58,600 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691398498600"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1691398498600"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1691398498600"}]},"ts":"1691398498600"} 2023-08-07 08:54:58,600 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1691398498600"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398498600"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398498600"}]},"ts":"1691398498600"} 2023-08-07 08:54:58,600 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1691398498600"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398498600"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398498600"}]},"ts":"1691398498600"} 2023-08-07 08:54:58,602 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:54:58,602 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:58,602 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:54:58,602 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:54:58,849 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:58,849 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:54:58,849 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:54:58,849 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:54:58,849 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 08:54:58,849 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:54:58,849 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:54:58,849 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:54:58,849 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:54:58,850 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:54:58,851 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-08-07 08:54:58,851 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-07 08:54:58,852 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-07 08:54:58,852 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:54:58,852 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:54:58,852 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:54:58,853 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:54:58,853 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:54:58,853 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:54:58,853 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:54:58,854 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:54:58,854 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:54:58,854 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:54:58,854 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-07 08:54:58,854 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-07 08:54:58,855 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:54:58,855 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:54:58,856 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN 2023-08-07 08:54:58,856 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:54:58,856 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:54:58,856 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:54:58,857 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=fd6d700fb2bf2b02eb0b469b2b0d401b, daughterA=9ca78a73cef89f541dcf91876dbfafd9, daughterB=5b3565e5e309ccbd325fa8dc34e561c4 2023-08-07 08:54:58,857 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:54:58,857 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:54:58,857 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:54:58,858 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0d0fd872f4eb265c927f025c1f8dd4da, daughterA=890310475de9dc263dd17491ca380a4c, daughterB=246cb6e862f1133bf06f7e32263c91b4 2023-08-07 08:54:58,858 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:54:58,858 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:54:58,858 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:58,859 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:58,859 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,34559,1691398396467, table=testRollbackForSplitTableRegionWithReplica, region=b0ddf60071e752aa19c5e065b70504c9 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=022268c9253618609543a9a81eddac57, daughterA=3088038bd36f4f725bfab047b7dd3b57, daughterB=60ab8c2a890a3fc5d5fda3e377b4c31f 2023-08-07 08:54:58,859 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:58,860 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:58,860 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:58,860 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:58,860 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:58,860 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-08-07 08:54:58,861 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:58,861 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=824d793ff4148484b7237e455699de76, UNASSIGN 2023-08-07 08:54:58,861 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 824d793ff4148484b7237e455699de76, server=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:58,861 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:54:58,861 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:58,861 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:58,862 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:58,862 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=824d793ff4148484b7237e455699de76, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:54:58,862 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 824d793ff4148484b7237e455699de76, server=jenkins-hbase4.apache.org,34559,1691398396467 didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:54:58,862 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 12 msec 2023-08-07 08:54:58,863 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:54:58,863 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:54:58,866 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398528866 2023-08-07 08:54:58,868 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:58,868 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:54:58,870 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:54:58,871 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=824d793ff4148484b7237e455699de76, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34559,1691398396467, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467, openSeqNum=2 2023-08-07 08:54:58,871 INFO [Listener at localhost/33561] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,34559,1691398396467, table=testRecoveryAndDoubleExecution, region=824d793ff4148484b7237e455699de76 2023-08-07 08:54:58,871 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=1ab5135a72784a2f50695787b50c8cb4, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-08-07 08:54:58,871 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=0618a943117f003311dce8de6834e559, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-08-07 08:54:58,871 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:54:58,871 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398558871 2023-08-07 08:54:58,871 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398618871 2023-08-07 08:54:58,871 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-08-07 08:54:58,872 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691398498872"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1691398498872"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1691398498872"}]},"ts":"1691398498872"} 2023-08-07 08:54:58,872 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1691398498872"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398498872"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398498872"}]},"ts":"1691398498872"} 2023-08-07 08:54:58,872 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1691398498872"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398498872"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398498872"}]},"ts":"1691398498872"} 2023-08-07 08:54:58,873 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:54:58,875 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:54:58,875 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:58,875 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:54:58,875 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:54:59,122 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:59,122 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:54:59,122 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:54:59,122 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:54:59,122 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 08:54:59,122 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:54:59,122 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:54:59,122 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:54:59,122 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:54:59,123 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:54:59,124 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-08-07 08:54:59,124 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-07 08:54:59,124 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-07 08:54:59,124 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:54:59,125 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:54:59,125 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:54:59,125 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:54:59,125 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:54:59,125 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:54:59,126 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:54:59,126 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:54:59,126 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:54:59,126 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:54:59,126 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-07 08:54:59,126 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-07 08:54:59,126 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:54:59,127 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:54:59,127 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN 2023-08-07 08:54:59,127 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:54:59,127 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:54:59,127 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:54:59,128 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=fd6d700fb2bf2b02eb0b469b2b0d401b, daughterA=9ca78a73cef89f541dcf91876dbfafd9, daughterB=5b3565e5e309ccbd325fa8dc34e561c4 2023-08-07 08:54:59,128 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:54:59,128 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:54:59,128 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:54:59,128 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0d0fd872f4eb265c927f025c1f8dd4da, daughterA=890310475de9dc263dd17491ca380a4c, daughterB=246cb6e862f1133bf06f7e32263c91b4 2023-08-07 08:54:59,128 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:54:59,129 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:54:59,129 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:59,129 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:59,129 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,34559,1691398396467, table=testRollbackForSplitTableRegionWithReplica, region=b0ddf60071e752aa19c5e065b70504c9 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=022268c9253618609543a9a81eddac57, daughterA=3088038bd36f4f725bfab047b7dd3b57, daughterB=60ab8c2a890a3fc5d5fda3e377b4c31f 2023-08-07 08:54:59,130 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:59,130 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:59,130 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:59,130 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:59,130 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:59,131 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-08-07 08:54:59,131 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:59,131 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=824d793ff4148484b7237e455699de76, UNASSIGN 2023-08-07 08:54:59,131 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 824d793ff4148484b7237e455699de76, server=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:59,131 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:54:59,132 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:59,132 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:59,132 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:59,132 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=824d793ff4148484b7237e455699de76, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:54:59,132 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 824d793ff4148484b7237e455699de76, server=jenkins-hbase4.apache.org,34559,1691398396467 didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:54:59,132 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-08-07 08:54:59,133 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:54:59,133 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:54:59,133 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398529133 2023-08-07 08:54:59,134 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:54:59,135 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:59,137 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:54:59,138 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=824d793ff4148484b7237e455699de76, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34559,1691398396467, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467, openSeqNum=2 2023-08-07 08:54:59,138 INFO [Listener at localhost/33561] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,34559,1691398396467, table=testRecoveryAndDoubleExecution, region=824d793ff4148484b7237e455699de76 2023-08-07 08:54:59,138 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=1ab5135a72784a2f50695787b50c8cb4, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-08-07 08:54:59,138 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=0618a943117f003311dce8de6834e559, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-08-07 08:54:59,138 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:54:59,138 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398559138 2023-08-07 08:54:59,138 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398619138 2023-08-07 08:54:59,138 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-07 08:54:59,138 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:54:59,140 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1ab5135a72784a2f50695787b50c8cb4, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0618a943117f003311dce8de6834e559, ASSIGN}] 2023-08-07 08:54:59,141 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:54:59,141 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:59,141 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:54:59,141 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:54:59,388 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:59,388 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:54:59,389 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:54:59,389 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:54:59,389 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 08:54:59,389 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:54:59,389 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:54:59,389 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:54:59,389 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:54:59,390 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:54:59,391 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-08-07 08:54:59,391 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-07 08:54:59,391 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-07 08:54:59,391 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:54:59,392 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:54:59,392 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:54:59,392 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:54:59,392 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:54:59,392 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:54:59,393 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:54:59,393 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:54:59,393 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:54:59,393 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:54:59,393 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-07 08:54:59,393 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-07 08:54:59,393 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:54:59,394 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:54:59,394 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN 2023-08-07 08:54:59,394 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:54:59,394 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:54:59,394 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:54:59,394 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=fd6d700fb2bf2b02eb0b469b2b0d401b, daughterA=9ca78a73cef89f541dcf91876dbfafd9, daughterB=5b3565e5e309ccbd325fa8dc34e561c4 2023-08-07 08:54:59,394 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:54:59,394 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:54:59,395 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:54:59,395 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0d0fd872f4eb265c927f025c1f8dd4da, daughterA=890310475de9dc263dd17491ca380a4c, daughterB=246cb6e862f1133bf06f7e32263c91b4 2023-08-07 08:54:59,395 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:54:59,395 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:54:59,395 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:59,395 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:59,395 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,34559,1691398396467, table=testRollbackForSplitTableRegionWithReplica, region=b0ddf60071e752aa19c5e065b70504c9 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=022268c9253618609543a9a81eddac57, daughterA=3088038bd36f4f725bfab047b7dd3b57, daughterB=60ab8c2a890a3fc5d5fda3e377b4c31f 2023-08-07 08:54:59,396 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:59,396 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:59,396 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:59,396 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:59,396 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:59,396 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-08-07 08:54:59,396 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:59,396 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=824d793ff4148484b7237e455699de76, UNASSIGN 2023-08-07 08:54:59,396 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 824d793ff4148484b7237e455699de76, server=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:59,397 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:54:59,397 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:59,397 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:59,397 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:59,397 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=824d793ff4148484b7237e455699de76, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:54:59,397 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 824d793ff4148484b7237e455699de76, server=jenkins-hbase4.apache.org,34559,1691398396467 didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:54:59,397 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-08-07 08:54:59,398 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:54:59,398 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:54:59,398 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398529398 2023-08-07 08:54:59,398 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:54:59,400 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:59,402 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:54:59,403 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=824d793ff4148484b7237e455699de76, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34559,1691398396467, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467, openSeqNum=2 2023-08-07 08:54:59,403 INFO [Listener at localhost/33561] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,34559,1691398396467, table=testRecoveryAndDoubleExecution, region=824d793ff4148484b7237e455699de76 2023-08-07 08:54:59,403 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=1ab5135a72784a2f50695787b50c8cb4, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-08-07 08:54:59,403 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=0618a943117f003311dce8de6834e559, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-08-07 08:54:59,403 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:54:59,403 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398559403 2023-08-07 08:54:59,403 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398619403 2023-08-07 08:54:59,403 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-08-07 08:54:59,403 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1ab5135a72784a2f50695787b50c8cb4, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0618a943117f003311dce8de6834e559, ASSIGN}] 2023-08-07 08:54:59,403 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:54:59,406 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0618a943117f003311dce8de6834e559, ASSIGN 2023-08-07 08:54:59,406 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0618a943117f003311dce8de6834e559, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,34559,1691398396467; forceNewPlan=false, retain=false 2023-08-07 08:54:59,407 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1ab5135a72784a2f50695787b50c8cb4, ASSIGN 2023-08-07 08:54:59,407 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1ab5135a72784a2f50695787b50c8cb4, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,34559,1691398396467; forceNewPlan=false, retain=false 2023-08-07 08:54:59,557 INFO [jenkins-hbase4:33357] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-08-07 08:54:59,558 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=1ab5135a72784a2f50695787b50c8cb4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:59,558 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1691398499558"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398499558"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398499558"}]},"ts":"1691398499558"} 2023-08-07 08:54:59,560 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=115, state=RUNNABLE; OpenRegionProcedure 1ab5135a72784a2f50695787b50c8cb4, server=jenkins-hbase4.apache.org,34559,1691398396467}] 2023-08-07 08:54:59,562 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=0618a943117f003311dce8de6834e559, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:59,562 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1691398499562"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398499562"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398499562"}]},"ts":"1691398499562"} 2023-08-07 08:54:59,563 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=116, state=RUNNABLE; OpenRegionProcedure 0618a943117f003311dce8de6834e559, server=jenkins-hbase4.apache.org,34559,1691398396467}] 2023-08-07 08:54:59,714 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4. 2023-08-07 08:54:59,714 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1ab5135a72784a2f50695787b50c8cb4, NAME => 'testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4.', STARTKEY => '', ENDKEY => '41'} 2023-08-07 08:54:59,714 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-07 08:54:59,714 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 1ab5135a72784a2f50695787b50c8cb4 2023-08-07 08:54:59,715 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:54:59,715 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1ab5135a72784a2f50695787b50c8cb4 2023-08-07 08:54:59,715 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1ab5135a72784a2f50695787b50c8cb4 2023-08-07 08:54:59,716 INFO [StoreOpener-1ab5135a72784a2f50695787b50c8cb4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 1ab5135a72784a2f50695787b50c8cb4 2023-08-07 08:54:59,716 DEBUG [StoreOpener-1ab5135a72784a2f50695787b50c8cb4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1 2023-08-07 08:54:59,717 DEBUG [StoreOpener-1ab5135a72784a2f50695787b50c8cb4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1 2023-08-07 08:54:59,717 INFO [StoreOpener-1ab5135a72784a2f50695787b50c8cb4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1ab5135a72784a2f50695787b50c8cb4 columnFamilyName cf1 2023-08-07 08:54:59,730 DEBUG [StoreOpener-1ab5135a72784a2f50695787b50c8cb4-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/002fdccfeb544d5ebf4e03d1ff495ff9.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/002fdccfeb544d5ebf4e03d1ff495ff9-bottom 2023-08-07 08:54:59,734 DEBUG [StoreOpener-1ab5135a72784a2f50695787b50c8cb4-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/1827182aa65f48799fe839b1686c377a.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/1827182aa65f48799fe839b1686c377a-bottom 2023-08-07 08:54:59,737 DEBUG [StoreOpener-1ab5135a72784a2f50695787b50c8cb4-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/1c949736b8214e9dbfb50301075ce902.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/1c949736b8214e9dbfb50301075ce902-bottom 2023-08-07 08:54:59,741 DEBUG [StoreOpener-1ab5135a72784a2f50695787b50c8cb4-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/69d045a353d543a08852dc2740e7a53a.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/69d045a353d543a08852dc2740e7a53a-bottom 2023-08-07 08:54:59,744 DEBUG [StoreOpener-1ab5135a72784a2f50695787b50c8cb4-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/95295d00bddc425d9a6ecbaf12982e76.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/95295d00bddc425d9a6ecbaf12982e76-bottom 2023-08-07 08:54:59,747 DEBUG [StoreOpener-1ab5135a72784a2f50695787b50c8cb4-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/de1cf1c939b34080981ef8a51826a2fc.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/de1cf1c939b34080981ef8a51826a2fc-bottom 2023-08-07 08:54:59,751 DEBUG [StoreOpener-1ab5135a72784a2f50695787b50c8cb4-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/fefdc6befe6d48f3a37a99f82fbf9b2a.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/fefdc6befe6d48f3a37a99f82fbf9b2a-bottom 2023-08-07 08:54:59,751 INFO [StoreOpener-1ab5135a72784a2f50695787b50c8cb4-1] regionserver.HStore(310): Store=1ab5135a72784a2f50695787b50c8cb4/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:54:59,751 INFO [StoreOpener-1ab5135a72784a2f50695787b50c8cb4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 1ab5135a72784a2f50695787b50c8cb4 2023-08-07 08:54:59,752 DEBUG [StoreOpener-1ab5135a72784a2f50695787b50c8cb4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2 2023-08-07 08:54:59,752 DEBUG [StoreOpener-1ab5135a72784a2f50695787b50c8cb4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2 2023-08-07 08:54:59,752 INFO [StoreOpener-1ab5135a72784a2f50695787b50c8cb4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1ab5135a72784a2f50695787b50c8cb4 columnFamilyName cf2 2023-08-07 08:54:59,764 DEBUG [StoreOpener-1ab5135a72784a2f50695787b50c8cb4-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/10b188603bf243919435c2772898d8ac.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/10b188603bf243919435c2772898d8ac-bottom 2023-08-07 08:54:59,767 DEBUG [StoreOpener-1ab5135a72784a2f50695787b50c8cb4-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/5c6b91afece54b158ca8c1f81468b5c1.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/5c6b91afece54b158ca8c1f81468b5c1-bottom 2023-08-07 08:54:59,771 DEBUG [StoreOpener-1ab5135a72784a2f50695787b50c8cb4-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/62417b4c0b964993adddf5033e482896.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/62417b4c0b964993adddf5033e482896-bottom 2023-08-07 08:54:59,774 DEBUG [StoreOpener-1ab5135a72784a2f50695787b50c8cb4-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/8a3c47f4f5444f1b871568e51c61c322.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/8a3c47f4f5444f1b871568e51c61c322-bottom 2023-08-07 08:54:59,777 DEBUG [StoreOpener-1ab5135a72784a2f50695787b50c8cb4-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/9b3879532a474f5e99feae370e7a6833.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/9b3879532a474f5e99feae370e7a6833-bottom 2023-08-07 08:54:59,780 DEBUG [StoreOpener-1ab5135a72784a2f50695787b50c8cb4-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/deb22c69ab9048f1a49e20e6ce9257d8.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/deb22c69ab9048f1a49e20e6ce9257d8-bottom 2023-08-07 08:54:59,783 DEBUG [StoreOpener-1ab5135a72784a2f50695787b50c8cb4-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/f029eb62f11946f9b8e35a73edb12a9e.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/f029eb62f11946f9b8e35a73edb12a9e-bottom 2023-08-07 08:54:59,783 INFO [StoreOpener-1ab5135a72784a2f50695787b50c8cb4-1] regionserver.HStore(310): Store=1ab5135a72784a2f50695787b50c8cb4/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:54:59,784 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4 2023-08-07 08:54:59,785 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4 2023-08-07 08:54:59,786 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-07 08:54:59,787 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1ab5135a72784a2f50695787b50c8cb4 2023-08-07 08:54:59,788 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1ab5135a72784a2f50695787b50c8cb4; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61565233, jitterRate=-0.08260653913021088}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-07 08:54:59,788 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1ab5135a72784a2f50695787b50c8cb4: 2023-08-07 08:54:59,789 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4., pid=117, masterSystemTime=1691398499711 2023-08-07 08:54:59,789 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:54:59,790 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-07 08:54:59,790 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-07 08:54:59,790 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-07 08:54:59,790 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4. 2023-08-07 08:54:59,790 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4. 2023-08-07 08:54:59,790 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HStore(1912): 1ab5135a72784a2f50695787b50c8cb4/cf1 is initiating minor compaction (all files) 2023-08-07 08:54:59,790 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HStore(1912): 1ab5135a72784a2f50695787b50c8cb4/cf2 is initiating minor compaction (all files) 2023-08-07 08:54:59,790 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 1ab5135a72784a2f50695787b50c8cb4/cf1 in testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4. 2023-08-07 08:54:59,790 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 1ab5135a72784a2f50695787b50c8cb4/cf2 in testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4. 2023-08-07 08:54:59,791 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/62417b4c0b964993adddf5033e482896.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/62417b4c0b964993adddf5033e482896-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/deb22c69ab9048f1a49e20e6ce9257d8.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/deb22c69ab9048f1a49e20e6ce9257d8-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/9b3879532a474f5e99feae370e7a6833.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/9b3879532a474f5e99feae370e7a6833-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/10b188603bf243919435c2772898d8ac.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/10b188603bf243919435c2772898d8ac-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/f029eb62f11946f9b8e35a73edb12a9e.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/f029eb62f11946f9b8e35a73edb12a9e-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/5c6b91afece54b158ca8c1f81468b5c1.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/5c6b91afece54b158ca8c1f81468b5c1-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/8a3c47f4f5444f1b871568e51c61c322.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/8a3c47f4f5444f1b871568e51c61c322-bottom] into tmpdir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/.tmp, totalSize=34.9 K 2023-08-07 08:54:59,791 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/de1cf1c939b34080981ef8a51826a2fc.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/de1cf1c939b34080981ef8a51826a2fc-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/002fdccfeb544d5ebf4e03d1ff495ff9.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/002fdccfeb544d5ebf4e03d1ff495ff9-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/fefdc6befe6d48f3a37a99f82fbf9b2a.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/fefdc6befe6d48f3a37a99f82fbf9b2a-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/1827182aa65f48799fe839b1686c377a.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/1827182aa65f48799fe839b1686c377a-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/95295d00bddc425d9a6ecbaf12982e76.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/95295d00bddc425d9a6ecbaf12982e76-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/1c949736b8214e9dbfb50301075ce902.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/1c949736b8214e9dbfb50301075ce902-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/69d045a353d543a08852dc2740e7a53a.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/69d045a353d543a08852dc2740e7a53a-bottom] into tmpdir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/.tmp, totalSize=34.9 K 2023-08-07 08:54:59,791 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting 62417b4c0b964993adddf5033e482896.824d793ff4148484b7237e455699de76, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1691398493908 2023-08-07 08:54:59,791 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4. 2023-08-07 08:54:59,791 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4. 2023-08-07 08:54:59,791 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting de1cf1c939b34080981ef8a51826a2fc.824d793ff4148484b7237e455699de76, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1691398493908 2023-08-07 08:54:59,791 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559. 2023-08-07 08:54:59,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 0618a943117f003311dce8de6834e559, NAME => 'testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559.', STARTKEY => '41', ENDKEY => ''} 2023-08-07 08:54:59,792 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting deb22c69ab9048f1a49e20e6ce9257d8.824d793ff4148484b7237e455699de76, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1691398494077 2023-08-07 08:54:59,792 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=1ab5135a72784a2f50695787b50c8cb4, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:59,792 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting 002fdccfeb544d5ebf4e03d1ff495ff9.824d793ff4148484b7237e455699de76, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1691398494077 2023-08-07 08:54:59,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-07 08:54:59,792 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1691398499792"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691398499792"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691398499792"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398499792"}]},"ts":"1691398499792"} 2023-08-07 08:54:59,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 0618a943117f003311dce8de6834e559 2023-08-07 08:54:59,792 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting 9b3879532a474f5e99feae370e7a6833.824d793ff4148484b7237e455699de76, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1691398494252 2023-08-07 08:54:59,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:54:59,792 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting fefdc6befe6d48f3a37a99f82fbf9b2a.824d793ff4148484b7237e455699de76, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1691398494252 2023-08-07 08:54:59,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 0618a943117f003311dce8de6834e559 2023-08-07 08:54:59,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 0618a943117f003311dce8de6834e559 2023-08-07 08:54:59,795 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=115 2023-08-07 08:54:59,795 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 1ab5135a72784a2f50695787b50c8cb4, server=jenkins-hbase4.apache.org,34559,1691398396467 in 233 msec 2023-08-07 08:54:59,796 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1ab5135a72784a2f50695787b50c8cb4, ASSIGN in 392 msec 2023-08-07 08:54:59,799 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting 10b188603bf243919435c2772898d8ac.824d793ff4148484b7237e455699de76, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1691398494429 2023-08-07 08:54:59,799 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting 1827182aa65f48799fe839b1686c377a.824d793ff4148484b7237e455699de76, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1691398494429 2023-08-07 08:54:59,799 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting f029eb62f11946f9b8e35a73edb12a9e.824d793ff4148484b7237e455699de76, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1691398494619 2023-08-07 08:54:59,799 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting 95295d00bddc425d9a6ecbaf12982e76.824d793ff4148484b7237e455699de76, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1691398494619 2023-08-07 08:54:59,799 INFO [StoreOpener-0618a943117f003311dce8de6834e559-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0618a943117f003311dce8de6834e559 2023-08-07 08:54:59,799 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting 5c6b91afece54b158ca8c1f81468b5c1.824d793ff4148484b7237e455699de76, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1691398494795 2023-08-07 08:54:59,800 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting 1c949736b8214e9dbfb50301075ce902.824d793ff4148484b7237e455699de76, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1691398494795 2023-08-07 08:54:59,800 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting 8a3c47f4f5444f1b871568e51c61c322.824d793ff4148484b7237e455699de76, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1691398494973 2023-08-07 08:54:59,800 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting 69d045a353d543a08852dc2740e7a53a.824d793ff4148484b7237e455699de76, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1691398494973 2023-08-07 08:54:59,800 DEBUG [StoreOpener-0618a943117f003311dce8de6834e559-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1 2023-08-07 08:54:59,800 DEBUG [StoreOpener-0618a943117f003311dce8de6834e559-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1 2023-08-07 08:54:59,801 INFO [StoreOpener-0618a943117f003311dce8de6834e559-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0618a943117f003311dce8de6834e559 columnFamilyName cf1 2023-08-07 08:54:59,815 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] throttle.PressureAwareThroughputController(145): 1ab5135a72784a2f50695787b50c8cb4#cf2#compaction#106 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-08-07 08:54:59,815 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1ab5135a72784a2f50695787b50c8cb4#cf1#compaction#107 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-07 08:54:59,817 DEBUG [StoreOpener-0618a943117f003311dce8de6834e559-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/002fdccfeb544d5ebf4e03d1ff495ff9.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/002fdccfeb544d5ebf4e03d1ff495ff9-top 2023-08-07 08:54:59,820 DEBUG [StoreOpener-0618a943117f003311dce8de6834e559-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/1827182aa65f48799fe839b1686c377a.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/1827182aa65f48799fe839b1686c377a-top 2023-08-07 08:54:59,824 DEBUG [StoreOpener-0618a943117f003311dce8de6834e559-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/1c949736b8214e9dbfb50301075ce902.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/1c949736b8214e9dbfb50301075ce902-top 2023-08-07 08:54:59,831 DEBUG [StoreOpener-0618a943117f003311dce8de6834e559-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/69d045a353d543a08852dc2740e7a53a.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/69d045a353d543a08852dc2740e7a53a-top 2023-08-07 08:54:59,840 DEBUG [StoreOpener-0618a943117f003311dce8de6834e559-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/95295d00bddc425d9a6ecbaf12982e76.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/95295d00bddc425d9a6ecbaf12982e76-top 2023-08-07 08:54:59,841 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/.tmp/cf1/bdd954c7a967422eb41c57b0f3bd7500 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/bdd954c7a967422eb41c57b0f3bd7500 2023-08-07 08:54:59,846 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/.tmp/cf2/1facea824e1740d78b9cbd71e146c1aa as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/1facea824e1740d78b9cbd71e146c1aa 2023-08-07 08:54:59,847 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 1ab5135a72784a2f50695787b50c8cb4/cf1 of 1ab5135a72784a2f50695787b50c8cb4 into bdd954c7a967422eb41c57b0f3bd7500(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 08:54:59,847 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1ab5135a72784a2f50695787b50c8cb4: 2023-08-07 08:54:59,847 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4., storeName=1ab5135a72784a2f50695787b50c8cb4/cf1, priority=9, startTime=1691398499789; duration=0sec 2023-08-07 08:54:59,847 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:54:59,850 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 1ab5135a72784a2f50695787b50c8cb4/cf2 of 1ab5135a72784a2f50695787b50c8cb4 into 1facea824e1740d78b9cbd71e146c1aa(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 08:54:59,850 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1ab5135a72784a2f50695787b50c8cb4: 2023-08-07 08:54:59,850 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4., storeName=1ab5135a72784a2f50695787b50c8cb4/cf2, priority=9, startTime=1691398499790; duration=0sec 2023-08-07 08:54:59,850 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:54:59,855 DEBUG [StoreOpener-0618a943117f003311dce8de6834e559-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/de1cf1c939b34080981ef8a51826a2fc.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/de1cf1c939b34080981ef8a51826a2fc-top 2023-08-07 08:54:59,859 DEBUG [StoreOpener-0618a943117f003311dce8de6834e559-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/fefdc6befe6d48f3a37a99f82fbf9b2a.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/fefdc6befe6d48f3a37a99f82fbf9b2a-top 2023-08-07 08:54:59,859 INFO [StoreOpener-0618a943117f003311dce8de6834e559-1] regionserver.HStore(310): Store=0618a943117f003311dce8de6834e559/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:54:59,859 INFO [StoreOpener-0618a943117f003311dce8de6834e559-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 0618a943117f003311dce8de6834e559 2023-08-07 08:54:59,860 DEBUG [StoreOpener-0618a943117f003311dce8de6834e559-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2 2023-08-07 08:54:59,860 DEBUG [StoreOpener-0618a943117f003311dce8de6834e559-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2 2023-08-07 08:54:59,860 INFO [StoreOpener-0618a943117f003311dce8de6834e559-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0618a943117f003311dce8de6834e559 columnFamilyName cf2 2023-08-07 08:54:59,871 DEBUG [StoreOpener-0618a943117f003311dce8de6834e559-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/10b188603bf243919435c2772898d8ac.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/10b188603bf243919435c2772898d8ac-top 2023-08-07 08:54:59,874 DEBUG [StoreOpener-0618a943117f003311dce8de6834e559-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/5c6b91afece54b158ca8c1f81468b5c1.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/5c6b91afece54b158ca8c1f81468b5c1-top 2023-08-07 08:54:59,877 DEBUG [StoreOpener-0618a943117f003311dce8de6834e559-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/62417b4c0b964993adddf5033e482896.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/62417b4c0b964993adddf5033e482896-top 2023-08-07 08:54:59,879 DEBUG [StoreOpener-0618a943117f003311dce8de6834e559-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/8a3c47f4f5444f1b871568e51c61c322.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/8a3c47f4f5444f1b871568e51c61c322-top 2023-08-07 08:54:59,883 DEBUG [StoreOpener-0618a943117f003311dce8de6834e559-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/9b3879532a474f5e99feae370e7a6833.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/9b3879532a474f5e99feae370e7a6833-top 2023-08-07 08:54:59,886 DEBUG [StoreOpener-0618a943117f003311dce8de6834e559-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/deb22c69ab9048f1a49e20e6ce9257d8.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/deb22c69ab9048f1a49e20e6ce9257d8-top 2023-08-07 08:54:59,889 DEBUG [StoreOpener-0618a943117f003311dce8de6834e559-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/f029eb62f11946f9b8e35a73edb12a9e.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/f029eb62f11946f9b8e35a73edb12a9e-top 2023-08-07 08:54:59,889 INFO [StoreOpener-0618a943117f003311dce8de6834e559-1] regionserver.HStore(310): Store=0618a943117f003311dce8de6834e559/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:54:59,890 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559 2023-08-07 08:54:59,890 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559 2023-08-07 08:54:59,892 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-07 08:54:59,893 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 0618a943117f003311dce8de6834e559 2023-08-07 08:54:59,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 0618a943117f003311dce8de6834e559; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69897418, jitterRate=0.04155269265174866}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-07 08:54:59,894 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 0618a943117f003311dce8de6834e559: 2023-08-07 08:54:59,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559., pid=118, masterSystemTime=1691398499711 2023-08-07 08:54:59,894 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:54:59,895 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-07 08:54:59,895 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-07 08:54:59,895 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-07 08:54:59,895 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559. 2023-08-07 08:54:59,896 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HStore(1912): 0618a943117f003311dce8de6834e559/cf1 is initiating minor compaction (all files) 2023-08-07 08:54:59,896 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559. 2023-08-07 08:54:59,896 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HStore(1912): 0618a943117f003311dce8de6834e559/cf2 is initiating minor compaction (all files) 2023-08-07 08:54:59,896 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 0618a943117f003311dce8de6834e559/cf1 in testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559. 2023-08-07 08:54:59,896 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 0618a943117f003311dce8de6834e559/cf2 in testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559. 2023-08-07 08:54:59,896 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/62417b4c0b964993adddf5033e482896.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/62417b4c0b964993adddf5033e482896-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/deb22c69ab9048f1a49e20e6ce9257d8.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/deb22c69ab9048f1a49e20e6ce9257d8-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/9b3879532a474f5e99feae370e7a6833.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/9b3879532a474f5e99feae370e7a6833-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/10b188603bf243919435c2772898d8ac.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/10b188603bf243919435c2772898d8ac-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/f029eb62f11946f9b8e35a73edb12a9e.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/f029eb62f11946f9b8e35a73edb12a9e-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/5c6b91afece54b158ca8c1f81468b5c1.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/5c6b91afece54b158ca8c1f81468b5c1-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/8a3c47f4f5444f1b871568e51c61c322.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/8a3c47f4f5444f1b871568e51c61c322-top] into tmpdir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/.tmp, totalSize=34.9 K 2023-08-07 08:54:59,896 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/de1cf1c939b34080981ef8a51826a2fc.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/de1cf1c939b34080981ef8a51826a2fc-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/002fdccfeb544d5ebf4e03d1ff495ff9.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/002fdccfeb544d5ebf4e03d1ff495ff9-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/fefdc6befe6d48f3a37a99f82fbf9b2a.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/fefdc6befe6d48f3a37a99f82fbf9b2a-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/1827182aa65f48799fe839b1686c377a.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/1827182aa65f48799fe839b1686c377a-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/95295d00bddc425d9a6ecbaf12982e76.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/95295d00bddc425d9a6ecbaf12982e76-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/1c949736b8214e9dbfb50301075ce902.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/1c949736b8214e9dbfb50301075ce902-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/69d045a353d543a08852dc2740e7a53a.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/69d045a353d543a08852dc2740e7a53a-top] into tmpdir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/.tmp, totalSize=34.9 K 2023-08-07 08:54:59,897 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting 62417b4c0b964993adddf5033e482896.824d793ff4148484b7237e455699de76, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1691398493908 2023-08-07 08:54:59,897 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting de1cf1c939b34080981ef8a51826a2fc.824d793ff4148484b7237e455699de76, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1691398493908 2023-08-07 08:54:59,897 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting deb22c69ab9048f1a49e20e6ce9257d8.824d793ff4148484b7237e455699de76, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1691398494077 2023-08-07 08:54:59,897 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559. 2023-08-07 08:54:59,897 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559. 2023-08-07 08:54:59,898 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting 002fdccfeb544d5ebf4e03d1ff495ff9.824d793ff4148484b7237e455699de76, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1691398494077 2023-08-07 08:54:59,898 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting 9b3879532a474f5e99feae370e7a6833.824d793ff4148484b7237e455699de76, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1691398494252 2023-08-07 08:54:59,898 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=0618a943117f003311dce8de6834e559, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:59,898 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting fefdc6befe6d48f3a37a99f82fbf9b2a.824d793ff4148484b7237e455699de76, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1691398494252 2023-08-07 08:54:59,898 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1691398499898"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691398499898"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691398499898"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398499898"}]},"ts":"1691398499898"} 2023-08-07 08:54:59,898 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting 10b188603bf243919435c2772898d8ac.824d793ff4148484b7237e455699de76, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1691398494429 2023-08-07 08:54:59,899 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting 1827182aa65f48799fe839b1686c377a.824d793ff4148484b7237e455699de76, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1691398494429 2023-08-07 08:54:59,899 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting f029eb62f11946f9b8e35a73edb12a9e.824d793ff4148484b7237e455699de76, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1691398494619 2023-08-07 08:54:59,899 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting 95295d00bddc425d9a6ecbaf12982e76.824d793ff4148484b7237e455699de76, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1691398494619 2023-08-07 08:54:59,900 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting 5c6b91afece54b158ca8c1f81468b5c1.824d793ff4148484b7237e455699de76, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1691398494795 2023-08-07 08:54:59,900 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting 1c949736b8214e9dbfb50301075ce902.824d793ff4148484b7237e455699de76, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1691398494795 2023-08-07 08:54:59,900 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting 8a3c47f4f5444f1b871568e51c61c322.824d793ff4148484b7237e455699de76, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1691398494973 2023-08-07 08:54:59,900 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting 69d045a353d543a08852dc2740e7a53a.824d793ff4148484b7237e455699de76, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1691398494973 2023-08-07 08:54:59,901 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=116 2023-08-07 08:54:59,901 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 0618a943117f003311dce8de6834e559, server=jenkins-hbase4.apache.org,34559,1691398396467 in 336 msec 2023-08-07 08:54:59,903 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=116, resume processing ppid=112 2023-08-07 08:54:59,903 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0618a943117f003311dce8de6834e559, ASSIGN in 498 msec 2023-08-07 08:54:59,903 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-07 08:54:59,903 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:59,903 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:54:59,903 INFO [Listener at localhost/33561] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:59,903 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-07 08:54:59,904 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-07 08:54:59,904 INFO [Listener at localhost/33561] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:54:59,904 INFO [Listener at localhost/33561] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:54:59,904 ERROR [Listener at localhost/33561] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 08:54:59,904 INFO [Listener at localhost/33561] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-07 08:54:59,904 INFO [Listener at localhost/33561] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-07 08:54:59,904 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-07 08:54:59,904 INFO [Listener at localhost/33561] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 08:54:59,905 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 08:54:59,907 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-08-07 08:54:59,907 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-07 08:54:59,907 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-07 08:54:59,907 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 16, 21] 2023-08-07 08:54:59,907 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 20] 2023-08-07 08:54:59,907 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 18, 23] 2023-08-07 08:54:59,907 DEBUG [Listener at localhost/33561] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 22] 2023-08-07 08:54:59,907 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 08:54:59,908 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 08:54:59,908 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 08:54:59,908 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 08:54:59,908 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:54:59,909 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=769ddd8aa61f13dd3dc139c4fa3762cf, daughterA=46ac7a76f9146bf6f12230cccd2abdee, daughterB=1d86003e731a97a463196e40fae27785 2023-08-07 08:54:59,909 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:54:59,909 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-07 08:54:59,909 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-07 08:54:59,910 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7092830e4beb39626d41e32e9a87a8fd, daughterA=e32a11e61a96e60ae4944e49d40a04ff, daughterB=a94b8879caa03f00060c928070e3e2ab 2023-08-07 08:54:59,910 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-07 08:54:59,910 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-07 08:54:59,910 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:54:59,911 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=fd83112e7ec055aba3e5a554dab2841e, daughterA=48c2ad6fc762203958def153b6e206ef, daughterB=1eb65ab1dd6aaf4ea2b55584fc60246a 2023-08-07 08:54:59,911 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=fd83112e7ec055aba3e5a554dab2841e, ASSIGN 2023-08-07 08:54:59,911 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:54:59,911 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-07 08:54:59,911 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:54:59,912 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=fd6d700fb2bf2b02eb0b469b2b0d401b, daughterA=9ca78a73cef89f541dcf91876dbfafd9, daughterB=5b3565e5e309ccbd325fa8dc34e561c4 2023-08-07 08:54:59,912 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:54:59,912 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-07 08:54:59,912 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:54:59,912 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0d0fd872f4eb265c927f025c1f8dd4da, daughterA=890310475de9dc263dd17491ca380a4c, daughterB=246cb6e862f1133bf06f7e32263c91b4 2023-08-07 08:54:59,912 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:54:59,913 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-07 08:54:59,913 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:59,913 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:59,913 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,34559,1691398396467, table=testRollbackForSplitTableRegionWithReplica, region=b0ddf60071e752aa19c5e065b70504c9 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=022268c9253618609543a9a81eddac57, daughterA=3088038bd36f4f725bfab047b7dd3b57, daughterB=60ab8c2a890a3fc5d5fda3e377b4c31f 2023-08-07 08:54:59,913 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:59,914 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-07 08:54:59,914 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:59,914 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:59,914 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-08-07 08:54:59,914 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-08-07 08:54:59,915 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] throttle.PressureAwareThroughputController(145): 0618a943117f003311dce8de6834e559#cf2#compaction#108 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-07 08:54:59,916 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:59,916 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=824d793ff4148484b7237e455699de76, UNASSIGN 2023-08-07 08:54:59,916 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 824d793ff4148484b7237e455699de76, server=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:59,916 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1ab5135a72784a2f50695787b50c8cb4, ASSIGN 2023-08-07 08:54:59,916 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0618a943117f003311dce8de6834e559, ASSIGN 2023-08-07 08:54:59,916 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 1ab5135a72784a2f50695787b50c8cb4, server=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:59,916 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(468): Loading pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 0618a943117f003311dce8de6834e559, server=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:54:59,916 DEBUG [Listener at localhost/33561] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 held the lock before restarting, call acquireLock to restore it. 2023-08-07 08:54:59,917 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:59,917 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:59,917 INFO [Listener at localhost/33561] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:59,917 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=824d793ff4148484b7237e455699de76, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:54:59,917 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 824d793ff4148484b7237e455699de76, server=jenkins-hbase4.apache.org,34559,1691398396467 didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:54:59,917 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1ab5135a72784a2f50695787b50c8cb4, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:54:59,917 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0618a943117f003311dce8de6834e559, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:54:59,917 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 1ab5135a72784a2f50695787b50c8cb4, server=jenkins-hbase4.apache.org,34559,1691398396467 didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:54:59,917 DEBUG [Listener at localhost/33561] procedure2.Procedure(946): pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 0618a943117f003311dce8de6834e559, server=jenkins-hbase4.apache.org,34559,1691398396467 didn't hold the lock before restarting, skip acquiring lock. 2023-08-07 08:54:59,917 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 12 msec 2023-08-07 08:54:59,919 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] throttle.PressureAwareThroughputController(145): 0618a943117f003311dce8de6834e559#cf1#compaction#109 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-07 08:54:59,923 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 08:54:59,923 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467, table=hbase:meta, region=1588230740 2023-08-07 08:54:59,931 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691398529930 2023-08-07 08:54:59,938 DEBUG [Listener at localhost/33561] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 08:54:59,939 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 2023-08-07 08:54:59,943 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=261180939175fe040900ce2fa993a79f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46525,1691398396377, regionLocation=jenkins-hbase4.apache.org,46525,1691398396377, openSeqNum=2 2023-08-07 08:54:59,943 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=824d793ff4148484b7237e455699de76, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34559,1691398396467, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467, openSeqNum=2 2023-08-07 08:54:59,943 INFO [Listener at localhost/33561] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,34559,1691398396467, table=testRecoveryAndDoubleExecution, region=824d793ff4148484b7237e455699de76 2023-08-07 08:54:59,943 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=1ab5135a72784a2f50695787b50c8cb4, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34559,1691398396467, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467, openSeqNum=86 2023-08-07 08:54:59,943 DEBUG [Listener at localhost/33561] assignment.RegionStateStore(147): Load hbase:meta entry region=0618a943117f003311dce8de6834e559, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34559,1691398396467, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467, openSeqNum=86 2023-08-07 08:54:59,944 INFO [Listener at localhost/33561] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-07 08:54:59,944 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691398559944 2023-08-07 08:54:59,944 INFO [Listener at localhost/33561] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691398619944 2023-08-07 08:54:59,944 INFO [Listener at localhost/33561] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-08-07 08:54:59,944 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-07 08:54:59,950 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=824d793ff4148484b7237e455699de76, daughterA=1ab5135a72784a2f50695787b50c8cb4, daughterB=0618a943117f003311dce8de6834e559 in 4.7130 sec 2023-08-07 08:54:59,957 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/.tmp/cf2/ac89aa388bfa4d7cbcd86b93d1b976ea as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/ac89aa388bfa4d7cbcd86b93d1b976ea 2023-08-07 08:54:59,961 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0618a943117f003311dce8de6834e559/cf2 of 0618a943117f003311dce8de6834e559 into ac89aa388bfa4d7cbcd86b93d1b976ea(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 08:54:59,961 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 0618a943117f003311dce8de6834e559: 2023-08-07 08:54:59,961 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559., storeName=0618a943117f003311dce8de6834e559/cf2, priority=9, startTime=1691398499895; duration=0sec 2023-08-07 08:54:59,961 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:55:00,195 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34559] ipc.CallRunner(144): callId: 858 service: ClientService methodName: Scan size: 154 connection: 172.31.14.131:45148 deadline: 1691398560195, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76. is not online on jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:00,310 DEBUG [Listener at localhost/33561] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-08-07 08:55:00,316 INFO [Listener at localhost/33561] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-08-07 08:55:00,318 DEBUG [Listener at localhost/33561] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-08-07 08:55:00,337 WARN [Listener at localhost/33561] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-07 08:55:00,338 INFO [Listener at localhost/33561] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-08-07 08:55:00,339 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testRecoveryAndDoubleExecution 2023-08-07 08:55:00,339 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-08-07 08:55:00,341 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=119 2023-08-07 08:55:00,342 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398500342"}]},"ts":"1691398500342"} 2023-08-07 08:55:00,343 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-08-07 08:55:00,345 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-08-07 08:55:00,346 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1ab5135a72784a2f50695787b50c8cb4, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0618a943117f003311dce8de6834e559, UNASSIGN}] 2023-08-07 08:55:00,347 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0618a943117f003311dce8de6834e559, UNASSIGN 2023-08-07 08:55:00,347 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=0618a943117f003311dce8de6834e559, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:00,347 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1691398500347"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398500347"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398500347"}]},"ts":"1691398500347"} 2023-08-07 08:55:00,349 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure 0618a943117f003311dce8de6834e559, server=jenkins-hbase4.apache.org,34559,1691398396467}] 2023-08-07 08:55:00,350 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/.tmp/cf1/dcde78c2a8dc48e9bd9e6b3abc0e81cd as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/dcde78c2a8dc48e9bd9e6b3abc0e81cd 2023-08-07 08:55:00,351 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1ab5135a72784a2f50695787b50c8cb4, UNASSIGN 2023-08-07 08:55:00,351 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=1ab5135a72784a2f50695787b50c8cb4, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:00,351 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1691398500351"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398500351"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398500351"}]},"ts":"1691398500351"} 2023-08-07 08:55:00,352 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure 1ab5135a72784a2f50695787b50c8cb4, server=jenkins-hbase4.apache.org,34559,1691398396467}] 2023-08-07 08:55:00,355 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0618a943117f003311dce8de6834e559/cf1 of 0618a943117f003311dce8de6834e559 into dcde78c2a8dc48e9bd9e6b3abc0e81cd(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 08:55:00,355 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 0618a943117f003311dce8de6834e559: 2023-08-07 08:55:00,355 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559., storeName=0618a943117f003311dce8de6834e559/cf1, priority=9, startTime=1691398499894; duration=0sec 2023-08-07 08:55:00,355 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:55:00,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=119 2023-08-07 08:55:00,501 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 0618a943117f003311dce8de6834e559 2023-08-07 08:55:00,501 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 0618a943117f003311dce8de6834e559, disabling compactions & flushes 2023-08-07 08:55:00,501 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559. 2023-08-07 08:55:00,501 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559. 2023-08-07 08:55:00,501 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559. after waiting 0 ms 2023-08-07 08:55:00,501 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559. 2023-08-07 08:55:00,501 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/de1cf1c939b34080981ef8a51826a2fc.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/de1cf1c939b34080981ef8a51826a2fc-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/002fdccfeb544d5ebf4e03d1ff495ff9.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/002fdccfeb544d5ebf4e03d1ff495ff9-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/fefdc6befe6d48f3a37a99f82fbf9b2a.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/fefdc6befe6d48f3a37a99f82fbf9b2a-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/1827182aa65f48799fe839b1686c377a.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/1827182aa65f48799fe839b1686c377a-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/95295d00bddc425d9a6ecbaf12982e76.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/95295d00bddc425d9a6ecbaf12982e76-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/1c949736b8214e9dbfb50301075ce902.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/1c949736b8214e9dbfb50301075ce902-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/69d045a353d543a08852dc2740e7a53a.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/69d045a353d543a08852dc2740e7a53a-top] to archive 2023-08-07 08:55:00,502 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-07 08:55:00,504 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/de1cf1c939b34080981ef8a51826a2fc.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/de1cf1c939b34080981ef8a51826a2fc.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,505 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/002fdccfeb544d5ebf4e03d1ff495ff9.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/002fdccfeb544d5ebf4e03d1ff495ff9.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,506 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/fefdc6befe6d48f3a37a99f82fbf9b2a.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/fefdc6befe6d48f3a37a99f82fbf9b2a.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,507 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/1827182aa65f48799fe839b1686c377a.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/1827182aa65f48799fe839b1686c377a.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,508 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/95295d00bddc425d9a6ecbaf12982e76.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/95295d00bddc425d9a6ecbaf12982e76.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,509 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/1c949736b8214e9dbfb50301075ce902.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/1c949736b8214e9dbfb50301075ce902.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,510 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/69d045a353d543a08852dc2740e7a53a.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/69d045a353d543a08852dc2740e7a53a.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,512 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/62417b4c0b964993adddf5033e482896.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/62417b4c0b964993adddf5033e482896-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/deb22c69ab9048f1a49e20e6ce9257d8.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/deb22c69ab9048f1a49e20e6ce9257d8-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/9b3879532a474f5e99feae370e7a6833.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/9b3879532a474f5e99feae370e7a6833-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/10b188603bf243919435c2772898d8ac.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/10b188603bf243919435c2772898d8ac-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/f029eb62f11946f9b8e35a73edb12a9e.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/f029eb62f11946f9b8e35a73edb12a9e-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/5c6b91afece54b158ca8c1f81468b5c1.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/5c6b91afece54b158ca8c1f81468b5c1-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/8a3c47f4f5444f1b871568e51c61c322.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/8a3c47f4f5444f1b871568e51c61c322-top] to archive 2023-08-07 08:55:00,512 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-07 08:55:00,514 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/62417b4c0b964993adddf5033e482896.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/62417b4c0b964993adddf5033e482896.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,515 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/deb22c69ab9048f1a49e20e6ce9257d8.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/deb22c69ab9048f1a49e20e6ce9257d8.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,516 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/9b3879532a474f5e99feae370e7a6833.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/9b3879532a474f5e99feae370e7a6833.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,517 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/10b188603bf243919435c2772898d8ac.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/10b188603bf243919435c2772898d8ac.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,518 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/f029eb62f11946f9b8e35a73edb12a9e.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/f029eb62f11946f9b8e35a73edb12a9e.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,520 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/5c6b91afece54b158ca8c1f81468b5c1.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/5c6b91afece54b158ca8c1f81468b5c1.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,521 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/8a3c47f4f5444f1b871568e51c61c322.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/8a3c47f4f5444f1b871568e51c61c322.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,524 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-08-07 08:55:00,525 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-07 08:55:00,525 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559. 2023-08-07 08:55:00,525 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 0618a943117f003311dce8de6834e559: 2023-08-07 08:55:00,527 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 0618a943117f003311dce8de6834e559 2023-08-07 08:55:00,527 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 1ab5135a72784a2f50695787b50c8cb4 2023-08-07 08:55:00,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1ab5135a72784a2f50695787b50c8cb4, disabling compactions & flushes 2023-08-07 08:55:00,527 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4. 2023-08-07 08:55:00,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4. 2023-08-07 08:55:00,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4. after waiting 0 ms 2023-08-07 08:55:00,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4. 2023-08-07 08:55:00,527 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=0618a943117f003311dce8de6834e559, regionState=CLOSED 2023-08-07 08:55:00,528 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1691398500527"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398500527"}]},"ts":"1691398500527"} 2023-08-07 08:55:00,528 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/de1cf1c939b34080981ef8a51826a2fc.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/de1cf1c939b34080981ef8a51826a2fc-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/002fdccfeb544d5ebf4e03d1ff495ff9.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/002fdccfeb544d5ebf4e03d1ff495ff9-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/fefdc6befe6d48f3a37a99f82fbf9b2a.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/fefdc6befe6d48f3a37a99f82fbf9b2a-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/1827182aa65f48799fe839b1686c377a.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/1827182aa65f48799fe839b1686c377a-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/95295d00bddc425d9a6ecbaf12982e76.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/95295d00bddc425d9a6ecbaf12982e76-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/1c949736b8214e9dbfb50301075ce902.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/1c949736b8214e9dbfb50301075ce902-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/69d045a353d543a08852dc2740e7a53a.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/69d045a353d543a08852dc2740e7a53a-bottom] to archive 2023-08-07 08:55:00,529 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-07 08:55:00,531 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/de1cf1c939b34080981ef8a51826a2fc.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/de1cf1c939b34080981ef8a51826a2fc.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,531 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-08-07 08:55:00,531 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure 0618a943117f003311dce8de6834e559, server=jenkins-hbase4.apache.org,34559,1691398396467 in 181 msec 2023-08-07 08:55:00,533 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0618a943117f003311dce8de6834e559, UNASSIGN in 186 msec 2023-08-07 08:55:00,533 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/002fdccfeb544d5ebf4e03d1ff495ff9.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/002fdccfeb544d5ebf4e03d1ff495ff9.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,534 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/fefdc6befe6d48f3a37a99f82fbf9b2a.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/fefdc6befe6d48f3a37a99f82fbf9b2a.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,535 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/1827182aa65f48799fe839b1686c377a.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/1827182aa65f48799fe839b1686c377a.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,536 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/95295d00bddc425d9a6ecbaf12982e76.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/95295d00bddc425d9a6ecbaf12982e76.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,537 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/1c949736b8214e9dbfb50301075ce902.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/1c949736b8214e9dbfb50301075ce902.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,539 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/69d045a353d543a08852dc2740e7a53a.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/69d045a353d543a08852dc2740e7a53a.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,541 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/62417b4c0b964993adddf5033e482896.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/62417b4c0b964993adddf5033e482896-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/deb22c69ab9048f1a49e20e6ce9257d8.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/deb22c69ab9048f1a49e20e6ce9257d8-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/9b3879532a474f5e99feae370e7a6833.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/9b3879532a474f5e99feae370e7a6833-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/10b188603bf243919435c2772898d8ac.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/10b188603bf243919435c2772898d8ac-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/f029eb62f11946f9b8e35a73edb12a9e.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/f029eb62f11946f9b8e35a73edb12a9e-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/5c6b91afece54b158ca8c1f81468b5c1.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/5c6b91afece54b158ca8c1f81468b5c1-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/8a3c47f4f5444f1b871568e51c61c322.824d793ff4148484b7237e455699de76->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/8a3c47f4f5444f1b871568e51c61c322-bottom] to archive 2023-08-07 08:55:00,541 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-07 08:55:00,543 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/62417b4c0b964993adddf5033e482896.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/62417b4c0b964993adddf5033e482896.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,544 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/deb22c69ab9048f1a49e20e6ce9257d8.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/deb22c69ab9048f1a49e20e6ce9257d8.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,545 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/9b3879532a474f5e99feae370e7a6833.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/9b3879532a474f5e99feae370e7a6833.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,546 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/10b188603bf243919435c2772898d8ac.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/10b188603bf243919435c2772898d8ac.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,547 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/f029eb62f11946f9b8e35a73edb12a9e.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/f029eb62f11946f9b8e35a73edb12a9e.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,548 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/5c6b91afece54b158ca8c1f81468b5c1.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/5c6b91afece54b158ca8c1f81468b5c1.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,549 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/8a3c47f4f5444f1b871568e51c61c322.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/8a3c47f4f5444f1b871568e51c61c322.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,552 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-08-07 08:55:00,552 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-07 08:55:00,553 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4. 2023-08-07 08:55:00,553 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1ab5135a72784a2f50695787b50c8cb4: 2023-08-07 08:55:00,555 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 1ab5135a72784a2f50695787b50c8cb4 2023-08-07 08:55:00,555 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=1ab5135a72784a2f50695787b50c8cb4, regionState=CLOSED 2023-08-07 08:55:00,555 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1691398500555"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398500555"}]},"ts":"1691398500555"} 2023-08-07 08:55:00,557 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=120 2023-08-07 08:55:00,557 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure 1ab5135a72784a2f50695787b50c8cb4, server=jenkins-hbase4.apache.org,34559,1691398396467 in 204 msec 2023-08-07 08:55:00,559 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=119 2023-08-07 08:55:00,559 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1ab5135a72784a2f50695787b50c8cb4, UNASSIGN in 212 msec 2023-08-07 08:55:00,559 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398500559"}]},"ts":"1691398500559"} 2023-08-07 08:55:00,560 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-08-07 08:55:00,562 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-08-07 08:55:00,563 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 223 msec 2023-08-07 08:55:00,644 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=119 2023-08-07 08:55:00,644 INFO [Listener at localhost/33561] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-08-07 08:55:00,644 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testRecoveryAndDoubleExecution 2023-08-07 08:55:00,645 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-08-07 08:55:00,647 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-08-07 08:55:00,648 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=124 2023-08-07 08:55:00,648 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-08-07 08:55:00,652 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,652 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4 2023-08-07 08:55:00,652 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559 2023-08-07 08:55:00,654 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/recovered.edits] 2023-08-07 08:55:00,654 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/1ab5135a72784a2f50695787b50c8cb4, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/recovered.edits] 2023-08-07 08:55:00,655 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/0618a943117f003311dce8de6834e559, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/recovered.edits] 2023-08-07 08:55:00,666 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/1ab5135a72784a2f50695787b50c8cb4/.regioninfo to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/1ab5135a72784a2f50695787b50c8cb4/.regioninfo 2023-08-07 08:55:00,667 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/002fdccfeb544d5ebf4e03d1ff495ff9 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/002fdccfeb544d5ebf4e03d1ff495ff9 2023-08-07 08:55:00,667 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/0618a943117f003311dce8de6834e559/.regioninfo to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/0618a943117f003311dce8de6834e559/.regioninfo 2023-08-07 08:55:00,668 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/1827182aa65f48799fe839b1686c377a to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/1827182aa65f48799fe839b1686c377a 2023-08-07 08:55:00,670 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/1ab5135a72784a2f50695787b50c8cb4/cf1/002fdccfeb544d5ebf4e03d1ff495ff9.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/1ab5135a72784a2f50695787b50c8cb4/cf1/002fdccfeb544d5ebf4e03d1ff495ff9.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,670 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/1c949736b8214e9dbfb50301075ce902 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/1c949736b8214e9dbfb50301075ce902 2023-08-07 08:55:00,670 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/0618a943117f003311dce8de6834e559/cf1/002fdccfeb544d5ebf4e03d1ff495ff9.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/0618a943117f003311dce8de6834e559/cf1/002fdccfeb544d5ebf4e03d1ff495ff9.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,671 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/1ab5135a72784a2f50695787b50c8cb4/cf1/1827182aa65f48799fe839b1686c377a.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/1ab5135a72784a2f50695787b50c8cb4/cf1/1827182aa65f48799fe839b1686c377a.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,671 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/69d045a353d543a08852dc2740e7a53a to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/69d045a353d543a08852dc2740e7a53a 2023-08-07 08:55:00,672 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/0618a943117f003311dce8de6834e559/cf1/1827182aa65f48799fe839b1686c377a.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/0618a943117f003311dce8de6834e559/cf1/1827182aa65f48799fe839b1686c377a.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,673 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/1ab5135a72784a2f50695787b50c8cb4/cf1/1c949736b8214e9dbfb50301075ce902.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/1ab5135a72784a2f50695787b50c8cb4/cf1/1c949736b8214e9dbfb50301075ce902.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,673 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/95295d00bddc425d9a6ecbaf12982e76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/95295d00bddc425d9a6ecbaf12982e76 2023-08-07 08:55:00,674 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/0618a943117f003311dce8de6834e559/cf1/1c949736b8214e9dbfb50301075ce902.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/0618a943117f003311dce8de6834e559/cf1/1c949736b8214e9dbfb50301075ce902.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,674 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/de1cf1c939b34080981ef8a51826a2fc to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/de1cf1c939b34080981ef8a51826a2fc 2023-08-07 08:55:00,675 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/1ab5135a72784a2f50695787b50c8cb4/cf1/69d045a353d543a08852dc2740e7a53a.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/1ab5135a72784a2f50695787b50c8cb4/cf1/69d045a353d543a08852dc2740e7a53a.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,675 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/0618a943117f003311dce8de6834e559/cf1/69d045a353d543a08852dc2740e7a53a.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/0618a943117f003311dce8de6834e559/cf1/69d045a353d543a08852dc2740e7a53a.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,676 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/fefdc6befe6d48f3a37a99f82fbf9b2a to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf1/fefdc6befe6d48f3a37a99f82fbf9b2a 2023-08-07 08:55:00,676 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/1ab5135a72784a2f50695787b50c8cb4/cf1/95295d00bddc425d9a6ecbaf12982e76.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/1ab5135a72784a2f50695787b50c8cb4/cf1/95295d00bddc425d9a6ecbaf12982e76.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,676 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/0618a943117f003311dce8de6834e559/cf1/95295d00bddc425d9a6ecbaf12982e76.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/0618a943117f003311dce8de6834e559/cf1/95295d00bddc425d9a6ecbaf12982e76.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,677 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/1ab5135a72784a2f50695787b50c8cb4/cf1/de1cf1c939b34080981ef8a51826a2fc.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/1ab5135a72784a2f50695787b50c8cb4/cf1/de1cf1c939b34080981ef8a51826a2fc.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,678 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/0618a943117f003311dce8de6834e559/cf1/de1cf1c939b34080981ef8a51826a2fc.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/0618a943117f003311dce8de6834e559/cf1/de1cf1c939b34080981ef8a51826a2fc.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,679 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/1ab5135a72784a2f50695787b50c8cb4/cf1/fefdc6befe6d48f3a37a99f82fbf9b2a.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/1ab5135a72784a2f50695787b50c8cb4/cf1/fefdc6befe6d48f3a37a99f82fbf9b2a.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,679 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/10b188603bf243919435c2772898d8ac to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/10b188603bf243919435c2772898d8ac 2023-08-07 08:55:00,680 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/0618a943117f003311dce8de6834e559/cf1/fefdc6befe6d48f3a37a99f82fbf9b2a.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/0618a943117f003311dce8de6834e559/cf1/fefdc6befe6d48f3a37a99f82fbf9b2a.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,680 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/5c6b91afece54b158ca8c1f81468b5c1 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/5c6b91afece54b158ca8c1f81468b5c1 2023-08-07 08:55:00,682 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/1ab5135a72784a2f50695787b50c8cb4/cf2/10b188603bf243919435c2772898d8ac.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/1ab5135a72784a2f50695787b50c8cb4/cf2/10b188603bf243919435c2772898d8ac.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,682 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/62417b4c0b964993adddf5033e482896 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/62417b4c0b964993adddf5033e482896 2023-08-07 08:55:00,683 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/8a3c47f4f5444f1b871568e51c61c322 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/8a3c47f4f5444f1b871568e51c61c322 2023-08-07 08:55:00,683 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/0618a943117f003311dce8de6834e559/cf2/10b188603bf243919435c2772898d8ac.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/0618a943117f003311dce8de6834e559/cf2/10b188603bf243919435c2772898d8ac.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,683 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/1ab5135a72784a2f50695787b50c8cb4/cf2/5c6b91afece54b158ca8c1f81468b5c1.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/1ab5135a72784a2f50695787b50c8cb4/cf2/5c6b91afece54b158ca8c1f81468b5c1.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,685 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/0618a943117f003311dce8de6834e559/cf2/5c6b91afece54b158ca8c1f81468b5c1.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/0618a943117f003311dce8de6834e559/cf2/5c6b91afece54b158ca8c1f81468b5c1.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,685 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/9b3879532a474f5e99feae370e7a6833 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/9b3879532a474f5e99feae370e7a6833 2023-08-07 08:55:00,685 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/1ab5135a72784a2f50695787b50c8cb4/cf2/62417b4c0b964993adddf5033e482896.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/1ab5135a72784a2f50695787b50c8cb4/cf2/62417b4c0b964993adddf5033e482896.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,686 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/0618a943117f003311dce8de6834e559/cf2/62417b4c0b964993adddf5033e482896.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/0618a943117f003311dce8de6834e559/cf2/62417b4c0b964993adddf5033e482896.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,686 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/1ab5135a72784a2f50695787b50c8cb4/cf2/8a3c47f4f5444f1b871568e51c61c322.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/1ab5135a72784a2f50695787b50c8cb4/cf2/8a3c47f4f5444f1b871568e51c61c322.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,687 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/deb22c69ab9048f1a49e20e6ce9257d8 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/deb22c69ab9048f1a49e20e6ce9257d8 2023-08-07 08:55:00,688 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/0618a943117f003311dce8de6834e559/cf2/8a3c47f4f5444f1b871568e51c61c322.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/0618a943117f003311dce8de6834e559/cf2/8a3c47f4f5444f1b871568e51c61c322.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,688 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/1ab5135a72784a2f50695787b50c8cb4/cf2/9b3879532a474f5e99feae370e7a6833.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/1ab5135a72784a2f50695787b50c8cb4/cf2/9b3879532a474f5e99feae370e7a6833.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,688 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/f029eb62f11946f9b8e35a73edb12a9e to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/cf2/f029eb62f11946f9b8e35a73edb12a9e 2023-08-07 08:55:00,689 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/0618a943117f003311dce8de6834e559/cf2/9b3879532a474f5e99feae370e7a6833.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/0618a943117f003311dce8de6834e559/cf2/9b3879532a474f5e99feae370e7a6833.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,689 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/1ab5135a72784a2f50695787b50c8cb4/cf2/deb22c69ab9048f1a49e20e6ce9257d8.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/1ab5135a72784a2f50695787b50c8cb4/cf2/deb22c69ab9048f1a49e20e6ce9257d8.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,690 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/0618a943117f003311dce8de6834e559/cf2/deb22c69ab9048f1a49e20e6ce9257d8.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/0618a943117f003311dce8de6834e559/cf2/deb22c69ab9048f1a49e20e6ce9257d8.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,691 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/1ab5135a72784a2f50695787b50c8cb4/cf2/f029eb62f11946f9b8e35a73edb12a9e.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/1ab5135a72784a2f50695787b50c8cb4/cf2/f029eb62f11946f9b8e35a73edb12a9e.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,691 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/recovered.edits/85.seqid to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76/recovered.edits/85.seqid 2023-08-07 08:55:00,692 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/0618a943117f003311dce8de6834e559/cf2/f029eb62f11946f9b8e35a73edb12a9e.824d793ff4148484b7237e455699de76 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/0618a943117f003311dce8de6834e559/cf2/f029eb62f11946f9b8e35a73edb12a9e.824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,692 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/824d793ff4148484b7237e455699de76 2023-08-07 08:55:00,693 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/bdd954c7a967422eb41c57b0f3bd7500 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf1/bdd954c7a967422eb41c57b0f3bd7500 2023-08-07 08:55:00,694 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/dcde78c2a8dc48e9bd9e6b3abc0e81cd to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf1/dcde78c2a8dc48e9bd9e6b3abc0e81cd 2023-08-07 08:55:00,696 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/1facea824e1740d78b9cbd71e146c1aa to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/cf2/1facea824e1740d78b9cbd71e146c1aa 2023-08-07 08:55:00,697 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/ac89aa388bfa4d7cbcd86b93d1b976ea to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/cf2/ac89aa388bfa4d7cbcd86b93d1b976ea 2023-08-07 08:55:00,699 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/recovered.edits/90.seqid to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4/recovered.edits/90.seqid 2023-08-07 08:55:00,699 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/1ab5135a72784a2f50695787b50c8cb4 2023-08-07 08:55:00,699 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/recovered.edits/90.seqid to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559/recovered.edits/90.seqid 2023-08-07 08:55:00,700 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testRecoveryAndDoubleExecution/0618a943117f003311dce8de6834e559 2023-08-07 08:55:00,700 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-08-07 08:55:00,702 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-08-07 08:55:00,704 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-08-07 08:55:00,706 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-08-07 08:55:00,706 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-08-07 08:55:00,707 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-08-07 08:55:00,707 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691398500707"}]},"ts":"9223372036854775807"} 2023-08-07 08:55:00,707 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691398500707"}]},"ts":"9223372036854775807"} 2023-08-07 08:55:00,707 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691398500707"}]},"ts":"9223372036854775807"} 2023-08-07 08:55:00,708 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-08-07 08:55:00,708 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 824d793ff4148484b7237e455699de76, NAME => 'testRecoveryAndDoubleExecution,,1691398493404.824d793ff4148484b7237e455699de76.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => 1ab5135a72784a2f50695787b50c8cb4, NAME => 'testRecoveryAndDoubleExecution,,1691398495231.1ab5135a72784a2f50695787b50c8cb4.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 0618a943117f003311dce8de6834e559, NAME => 'testRecoveryAndDoubleExecution,41,1691398495231.0618a943117f003311dce8de6834e559.', STARTKEY => '41', ENDKEY => ''}] 2023-08-07 08:55:00,708 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-08-07 08:55:00,709 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1691398500709"}]},"ts":"9223372036854775807"} 2023-08-07 08:55:00,710 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-08-07 08:55:00,712 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-08-07 08:55:00,713 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 67 msec 2023-08-07 08:55:00,748 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=124 2023-08-07 08:55:00,749 INFO [Listener at localhost/33561] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-08-07 08:55:00,763 INFO [Listener at localhost/33561] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=374 (was 367) - Thread LEAK? -, OpenFileDescriptor=836 (was 743) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=194 (was 202), ProcessCount=175 (was 175), AvailableMemoryMB=1806 (was 1820) 2023-08-07 08:55:00,778 INFO [Listener at localhost/33561] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=374, OpenFileDescriptor=836, MaxFileDescriptor=60000, SystemLoadAverage=194, ProcessCount=175, AvailableMemoryMB=1805 2023-08-07 08:55:00,778 WARN [Listener at localhost/33561] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-07 08:55:00,780 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-08-07 08:55:00,781 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-08-07 08:55:00,782 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(1029): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-08-07 08:55:00,784 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-08-07 08:55:00,784 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-07 08:55:00,786 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:00,786 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545 empty. 2023-08-07 08:55:00,787 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:00,787 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-08-07 08:55:00,795 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-08-07 08:55:00,796 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => 42c4fe66166a76787d86f5399d5eb545, NAME => 'testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp 2023-08-07 08:55:01,204 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:55:01,205 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1604): Closing 42c4fe66166a76787d86f5399d5eb545, disabling compactions & flushes 2023-08-07 08:55:01,205 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545. 2023-08-07 08:55:01,205 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545. 2023-08-07 08:55:01,205 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545. after waiting 0 ms 2023-08-07 08:55:01,205 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545. 2023-08-07 08:55:01,205 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545. 2023-08-07 08:55:01,205 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1558): Region close journal for 42c4fe66166a76787d86f5399d5eb545: 2023-08-07 08:55:01,207 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-08-07 08:55:01,208 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1691398501208"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398501208"}]},"ts":"1691398501208"} 2023-08-07 08:55:01,209 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-07 08:55:01,210 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-07 08:55:01,210 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398501210"}]},"ts":"1691398501210"} 2023-08-07 08:55:01,211 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-08-07 08:55:01,214 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-07 08:55:01,215 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-07 08:55:01,215 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-07 08:55:01,215 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-07 08:55:01,215 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-07 08:55:01,216 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=42c4fe66166a76787d86f5399d5eb545, ASSIGN}] 2023-08-07 08:55:01,217 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=42c4fe66166a76787d86f5399d5eb545, ASSIGN 2023-08-07 08:55:01,217 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=42c4fe66166a76787d86f5399d5eb545, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34633,1691398396427; forceNewPlan=false, retain=false 2023-08-07 08:55:01,368 INFO [jenkins-hbase4:33357] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-07 08:55:01,369 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=42c4fe66166a76787d86f5399d5eb545, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:01,369 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1691398501369"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398501369"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398501369"}]},"ts":"1691398501369"} 2023-08-07 08:55:01,370 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure 42c4fe66166a76787d86f5399d5eb545, server=jenkins-hbase4.apache.org,34633,1691398396427}] 2023-08-07 08:55:01,528 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545. 2023-08-07 08:55:01,529 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 42c4fe66166a76787d86f5399d5eb545, NAME => 'testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545.', STARTKEY => '', ENDKEY => ''} 2023-08-07 08:55:01,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-07 08:55:01,529 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:01,529 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:55:01,529 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:01,529 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:01,530 INFO [StoreOpener-42c4fe66166a76787d86f5399d5eb545-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:01,532 DEBUG [StoreOpener-42c4fe66166a76787d86f5399d5eb545-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1 2023-08-07 08:55:01,532 DEBUG [StoreOpener-42c4fe66166a76787d86f5399d5eb545-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1 2023-08-07 08:55:01,532 INFO [StoreOpener-42c4fe66166a76787d86f5399d5eb545-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 42c4fe66166a76787d86f5399d5eb545 columnFamilyName cf1 2023-08-07 08:55:01,532 INFO [StoreOpener-42c4fe66166a76787d86f5399d5eb545-1] regionserver.HStore(310): Store=42c4fe66166a76787d86f5399d5eb545/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:55:01,533 INFO [StoreOpener-42c4fe66166a76787d86f5399d5eb545-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:01,534 DEBUG [StoreOpener-42c4fe66166a76787d86f5399d5eb545-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2 2023-08-07 08:55:01,534 DEBUG [StoreOpener-42c4fe66166a76787d86f5399d5eb545-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2 2023-08-07 08:55:01,534 INFO [StoreOpener-42c4fe66166a76787d86f5399d5eb545-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 42c4fe66166a76787d86f5399d5eb545 columnFamilyName cf2 2023-08-07 08:55:01,535 INFO [StoreOpener-42c4fe66166a76787d86f5399d5eb545-1] regionserver.HStore(310): Store=42c4fe66166a76787d86f5399d5eb545/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:55:01,535 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:01,536 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:01,537 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-07 08:55:01,538 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:01,540 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 08:55:01,540 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 42c4fe66166a76787d86f5399d5eb545; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62430822, jitterRate=-0.06970825791358948}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-07 08:55:01,540 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 42c4fe66166a76787d86f5399d5eb545: 2023-08-07 08:55:01,541 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545., pid=127, masterSystemTime=1691398501521 2023-08-07 08:55:01,543 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545. 2023-08-07 08:55:01,543 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545. 2023-08-07 08:55:01,543 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=42c4fe66166a76787d86f5399d5eb545, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:01,543 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1691398501543"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691398501543"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691398501543"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398501543"}]},"ts":"1691398501543"} 2023-08-07 08:55:01,546 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=126 2023-08-07 08:55:01,546 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure 42c4fe66166a76787d86f5399d5eb545, server=jenkins-hbase4.apache.org,34633,1691398396427 in 174 msec 2023-08-07 08:55:01,547 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-08-07 08:55:01,547 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=42c4fe66166a76787d86f5399d5eb545, ASSIGN in 330 msec 2023-08-07 08:55:01,548 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-07 08:55:01,548 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398501548"}]},"ts":"1691398501548"} 2023-08-07 08:55:01,549 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-08-07 08:55:01,551 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-08-07 08:55:01,552 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 770 msec 2023-08-07 08:55:01,726 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-08-07 08:55:01,785 DEBUG [Listener at localhost/33561] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 08:55:01,789 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51646, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-07 08:55:01,794 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:55:01,795 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,795 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-08-07 08:55:01,795 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:01,795 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-08-07 08:55:01,796 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-08-07 08:55:01,796 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,796 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,797 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:01,797 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:01,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:01,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:01,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:01,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:01,797 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:01,797 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:01,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:01,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:01,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,798 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:01,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,798 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,798 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:55:01,798 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,798 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:55:01,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:55:01,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,798 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,798 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:55:01,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,799 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,799 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,799 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,799 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,799 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-07 08:55:01,799 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-07 08:55:01,799 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:01,799 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-07 08:55:01,799 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-07 08:55:01,799 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:01,799 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:01,799 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-07 08:55:01,799 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-07 08:55:01,799 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:55:01,799 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:55:01,799 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-07 08:55:01,799 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545. 2023-08-07 08:55:01,799 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-07 08:55:01,799 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545. started... 2023-08-07 08:55:01,799 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-07 08:55:01,799 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-07 08:55:01,800 INFO [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 42c4fe66166a76787d86f5399d5eb545 2/2 column families, dataSize=120 B heapSize=928 B 2023-08-07 08:55:01,803 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,803 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,803 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,803 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:55:01,803 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,803 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:55:01,812 INFO [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.tmp/cf1/1cd40e3a18e648c2bb75ad6f4918564d 2023-08-07 08:55:01,822 INFO [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.tmp/cf2/6c30eef80f2a43ec8cb3d368f83c2ae9 2023-08-07 08:55:01,826 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.tmp/cf1/1cd40e3a18e648c2bb75ad6f4918564d as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/1cd40e3a18e648c2bb75ad6f4918564d 2023-08-07 08:55:01,829 INFO [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/1cd40e3a18e648c2bb75ad6f4918564d, entries=2, sequenceid=6, filesize=4.8 K 2023-08-07 08:55:01,829 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.tmp/cf2/6c30eef80f2a43ec8cb3d368f83c2ae9 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/6c30eef80f2a43ec8cb3d368f83c2ae9 2023-08-07 08:55:01,832 INFO [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/6c30eef80f2a43ec8cb3d368f83c2ae9, entries=2, sequenceid=6, filesize=4.8 K 2023-08-07 08:55:01,833 INFO [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 42c4fe66166a76787d86f5399d5eb545 in 33ms, sequenceid=6, compaction requested=false 2023-08-07 08:55:01,833 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-08-07 08:55:01,833 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 42c4fe66166a76787d86f5399d5eb545: 2023-08-07 08:55:01,833 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545. 2023-08-07 08:55:01,833 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:55:01,833 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:55:01,834 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:01,834 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-07 08:55:01,834 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-07 08:55:01,835 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,835 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:01,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:01,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:55:01,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:55:01,835 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,835 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:55:01,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:55:01,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:55:01,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:01,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:01,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:01,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:55:01,837 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-08-07 08:55:01,837 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@33aea5f7[Count = 0] remaining members to acquire global barrier 2023-08-07 08:55:01,837 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-08-07 08:55:01,837 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,838 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,838 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,838 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,839 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-07 08:55:01,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,839 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-07 08:55:01,839 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-07 08:55:01,839 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-07 08:55:01,839 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:55:01,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,839 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:55:01,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,839 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:01,839 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-07 08:55:01,839 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-07 08:55:01,839 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:55:01,839 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:55:01,840 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-07 08:55:01,840 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:01,840 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-07 08:55:01,842 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-07 08:55:01,842 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:01,842 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:01,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:01,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:55:01,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:55:01,842 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-07 08:55:01,843 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-07 08:55:01,843 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:01,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:55:01,843 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-07 08:55:01,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:55:01,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:01,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:01,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:01,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:55:01,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:01,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:01,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:01,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,34633,1691398396427': 2023-08-07 08:55:01,845 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-08-07 08:55:01,845 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-08-07 08:55:01,845 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:55:01,845 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:55:01,845 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,845 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:55:01,847 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,847 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,847 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,847 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:01,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:55:01,847 ERROR [BP-1940969354-172.31.14.131-1691398391279 heartbeating to localhost/127.0.0.1:45787] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1940969354-172.31.14.131-1691398391279 (Datanode Uuid 0712b291-8bcc-4f35-bb85-15469634c482) service to localhost/127.0.0.1:45787 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-08-07 08:55:01,847 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:01,847 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:01,847 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:01,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:55:01,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:01,848 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:01,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,848 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:01,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:01,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:01,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:55:01,848 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:01,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:01,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:01,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:55:01,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:01,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:01,850 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:01,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:01,850 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:01,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:55:01,850 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:01,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:01,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:01,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:01,861 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:01,862 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:01,861 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:01,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:01,862 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:01,862 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-08-07 08:55:01,862 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:55:01,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:01,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:01,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:01,862 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-08-07 08:55:01,862 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:01,862 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:01,862 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:01,862 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:01,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:01,862 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:01,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:01,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:01,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:01,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:01,862 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:55:01,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:01,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:01,862 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:01,863 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:01,863 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,863 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:01,863 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,863 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:01,863 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:01,863 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,863 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,963 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:55:01,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:55:01,975 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:55:01,976 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-08-07 08:55:01,976 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,976 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-08-07 08:55:01,976 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:01,976 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-08-07 08:55:01,976 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-08-07 08:55:01,977 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,977 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,978 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:01,978 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:01,978 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:01,978 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:01,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:01,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:01,978 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:01,979 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:55:01,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:01,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:01,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:01,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:01,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,979 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,979 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,979 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:55:01,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:55:01,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:55:01,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,980 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,980 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,980 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,980 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-07 08:55:01,980 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,980 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-07 08:55:01,980 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:01,980 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-07 08:55:01,980 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-07 08:55:01,980 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,980 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-07 08:55:01,980 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-07 08:55:01,980 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,980 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:01,980 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-07 08:55:01,981 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-07 08:55:01,981 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-07 08:55:01,981 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-07 08:55:01,981 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-07 08:55:01,981 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:01,981 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-07 08:55:01,981 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-07 08:55:01,981 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:55:01,981 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:55:01,981 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545. 2023-08-07 08:55:01,981 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545. started... 2023-08-07 08:55:01,981 INFO [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 42c4fe66166a76787d86f5399d5eb545 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:55:01,982 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,982 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,982 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,982 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:01,983 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:55:01,983 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:55:02,390 INFO [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.tmp/cf1/eb185e0905984e5eb44f41b140d6b93e 2023-08-07 08:55:02,402 INFO [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.tmp/cf2/693aae8c23f647cb8cc80322affd4413 2023-08-07 08:55:02,405 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.tmp/cf1/eb185e0905984e5eb44f41b140d6b93e as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/eb185e0905984e5eb44f41b140d6b93e 2023-08-07 08:55:02,408 INFO [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/eb185e0905984e5eb44f41b140d6b93e, entries=10, sequenceid=19, filesize=5.0 K 2023-08-07 08:55:02,409 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.tmp/cf2/693aae8c23f647cb8cc80322affd4413 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/693aae8c23f647cb8cc80322affd4413 2023-08-07 08:55:02,412 INFO [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/693aae8c23f647cb8cc80322affd4413, entries=10, sequenceid=19, filesize=5.0 K 2023-08-07 08:55:02,415 INFO [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 42c4fe66166a76787d86f5399d5eb545 in 434ms, sequenceid=19, compaction requested=false 2023-08-07 08:55:02,415 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 42c4fe66166a76787d86f5399d5eb545: 2023-08-07 08:55:02,415 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545. 2023-08-07 08:55:02,415 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:55:02,415 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:55:02,415 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,415 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-07 08:55:02,415 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-07 08:55:02,416 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,416 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:55:02,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:55:02,417 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,417 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:55:02,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:55:02,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:55:02,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:02,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:02,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:55:02,418 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-08-07 08:55:02,418 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@252cd127[Count = 0] remaining members to acquire global barrier 2023-08-07 08:55:02,418 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-08-07 08:55:02,419 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,420 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,420 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,420 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,421 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,421 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:55:02,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,421 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-07 08:55:02,421 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-07 08:55:02,421 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:55:02,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,421 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-07 08:55:02,421 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-07 08:55:02,421 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-07 08:55:02,421 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:55:02,421 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-07 08:55:02,421 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:55:02,422 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-07 08:55:02,422 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:02,422 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-07 08:55:02,423 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-07 08:55:02,423 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,423 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:02,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:55:02,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:55:02,423 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-07 08:55:02,424 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-07 08:55:02,424 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:02,424 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-07 08:55:02,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:55:02,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:55:02,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:02,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:02,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:55:02,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:02,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:02,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,34633,1691398396427': 2023-08-07 08:55:02,427 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-08-07 08:55:02,427 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-08-07 08:55:02,427 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:55:02,427 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:55:02,427 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,427 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:55:02,429 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,429 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,429 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:55:02,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:55:02,429 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,429 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:02,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,430 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:02,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:55:02,430 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:02,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:02,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,430 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:02,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:02,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:02,431 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:02,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:02,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:02,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:55:02,431 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:02,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:02,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:02,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:55:02,432 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,433 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:02,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,433 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:02,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:02,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:02,435 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:02,435 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:02,435 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:02,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:02,435 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:02,435 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:02,435 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:02,435 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:02,436 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:02,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:02,435 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-08-07 08:55:02,436 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:55:02,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:02,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:02,435 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:02,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:02,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:02,436 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-08-07 08:55:02,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:02,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:02,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:02,436 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,436 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:55:02,436 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:02,436 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:02,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:02,436 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:02,437 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,437 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:02,437 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,437 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,437 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,536 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:55:02,537 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:55:02,547 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:55:02,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-08-07 08:55:02,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,548 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-08-07 08:55:02,549 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:02,549 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-08-07 08:55:02,549 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-08-07 08:55:02,549 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,549 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,550 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:02,550 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:02,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:02,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:02,550 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,550 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:02,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:02,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:02,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:02,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:02,551 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,551 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:55:02,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,551 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,551 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,551 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:55:02,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:55:02,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:55:02,552 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,552 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,552 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,552 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-07 08:55:02,552 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,552 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-07 08:55:02,552 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-07 08:55:02,552 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:02,552 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,552 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-07 08:55:02,552 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-07 08:55:02,552 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:02,552 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-07 08:55:02,552 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-07 08:55:02,552 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,552 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-07 08:55:02,552 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-07 08:55:02,552 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:02,552 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-07 08:55:02,552 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-07 08:55:02,552 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-07 08:55:02,552 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-07 08:55:02,552 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:55:02,552 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:55:02,553 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545. 2023-08-07 08:55:02,553 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545. started... 2023-08-07 08:55:02,553 INFO [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 42c4fe66166a76787d86f5399d5eb545 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:55:02,554 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,555 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,555 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:55:02,555 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,556 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,556 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:55:02,563 INFO [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.tmp/cf1/ee816b081ce843248046865a7e7bd371 2023-08-07 08:55:02,576 INFO [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.tmp/cf2/a658b133e5f74726a921246fd9b66ab3 2023-08-07 08:55:02,579 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.tmp/cf1/ee816b081ce843248046865a7e7bd371 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/ee816b081ce843248046865a7e7bd371 2023-08-07 08:55:02,582 INFO [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/ee816b081ce843248046865a7e7bd371, entries=10, sequenceid=32, filesize=5.0 K 2023-08-07 08:55:02,583 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.tmp/cf2/a658b133e5f74726a921246fd9b66ab3 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/a658b133e5f74726a921246fd9b66ab3 2023-08-07 08:55:02,586 INFO [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/a658b133e5f74726a921246fd9b66ab3, entries=10, sequenceid=32, filesize=5.0 K 2023-08-07 08:55:02,588 INFO [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 42c4fe66166a76787d86f5399d5eb545 in 35ms, sequenceid=32, compaction requested=true 2023-08-07 08:55:02,588 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 42c4fe66166a76787d86f5399d5eb545: 2023-08-07 08:55:02,588 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545. 2023-08-07 08:55:02,588 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:55:02,588 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:55:02,588 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,588 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-07 08:55:02,588 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-07 08:55:02,590 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,590 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:55:02,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:55:02,590 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,590 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:55:02,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:55:02,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:55:02,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:02,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:02,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:55:02,592 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-08-07 08:55:02,592 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@edae81e[Count = 0] remaining members to acquire global barrier 2023-08-07 08:55:02,592 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-08-07 08:55:02,592 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,593 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,593 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,593 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,594 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-07 08:55:02,594 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,594 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:55:02,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,594 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-07 08:55:02,594 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-07 08:55:02,594 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-07 08:55:02,594 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-07 08:55:02,594 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:55:02,594 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-07 08:55:02,594 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:55:02,594 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:55:02,599 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-07 08:55:02,599 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:02,599 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-07 08:55:02,599 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:02,599 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-07 08:55:02,599 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-07 08:55:02,599 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-07 08:55:02,599 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:02,599 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,599 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-07 08:55:02,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:55:02,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:55:02,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:55:02,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:55:02,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:02,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:02,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:55:02,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:02,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:02,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,34633,1691398396427': 2023-08-07 08:55:02,602 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-08-07 08:55:02,602 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-08-07 08:55:02,602 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:55:02,602 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:55:02,602 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,603 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:55:02,604 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,604 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,604 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,604 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,604 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:02,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,604 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:02,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,604 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:02,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:55:02,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:55:02,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,605 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:02,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:02,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:02,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:02,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:02,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:55:02,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:02,605 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:02,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,606 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:02,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:55:02,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:02,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:02,607 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:02,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:55:02,607 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,608 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:02,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:02,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:02,610 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:02,610 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:02,610 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:02,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:02,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:02,610 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:02,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:02,610 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:02,610 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:02,610 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,611 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:02,611 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:02,611 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-08-07 08:55:02,611 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:55:02,610 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:02,611 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:02,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:02,611 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:02,611 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,611 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,611 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:02,611 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,611 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,611 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-08-07 08:55:02,611 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:02,611 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:55:02,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:02,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:02,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:02,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:02,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:02,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:02,711 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:55:02,712 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:55:02,723 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:55:02,724 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-08-07 08:55:02,724 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,724 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-08-07 08:55:02,724 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:02,725 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-08-07 08:55:02,725 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-08-07 08:55:02,725 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,725 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,727 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:02,727 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:02,727 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:02,727 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:02,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:02,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:02,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:02,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:02,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:02,727 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,727 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:55:02,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,728 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,728 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,728 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:55:02,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:55:02,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:55:02,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,729 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,729 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,729 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,729 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-07 08:55:02,729 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,729 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-07 08:55:02,729 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:02,729 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-07 08:55:02,729 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-07 08:55:02,729 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-07 08:55:02,730 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-07 08:55:02,730 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-07 08:55:02,730 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,730 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,730 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-07 08:55:02,730 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-07 08:55:02,730 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:02,730 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:02,730 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-07 08:55:02,730 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-07 08:55:02,730 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-07 08:55:02,730 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-07 08:55:02,731 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,731 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,731 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:55:02,732 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,732 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,732 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:55:02,734 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:55:02,734 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:55:02,734 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545. 2023-08-07 08:55:02,734 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545. started... 2023-08-07 08:55:02,734 INFO [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 42c4fe66166a76787d86f5399d5eb545 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:55:02,743 INFO [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.tmp/cf1/ced9e8182e354b89a6f3d6d205f31c07 2023-08-07 08:55:02,757 INFO [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.tmp/cf2/615d7da3ff9546eab7f2502413a6b1fa 2023-08-07 08:55:02,760 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.tmp/cf1/ced9e8182e354b89a6f3d6d205f31c07 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/ced9e8182e354b89a6f3d6d205f31c07 2023-08-07 08:55:02,763 INFO [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/ced9e8182e354b89a6f3d6d205f31c07, entries=10, sequenceid=45, filesize=5.0 K 2023-08-07 08:55:02,764 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.tmp/cf2/615d7da3ff9546eab7f2502413a6b1fa as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/615d7da3ff9546eab7f2502413a6b1fa 2023-08-07 08:55:02,767 INFO [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/615d7da3ff9546eab7f2502413a6b1fa, entries=10, sequenceid=45, filesize=5.0 K 2023-08-07 08:55:02,768 INFO [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 42c4fe66166a76787d86f5399d5eb545 in 34ms, sequenceid=45, compaction requested=true 2023-08-07 08:55:02,768 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 42c4fe66166a76787d86f5399d5eb545: 2023-08-07 08:55:02,768 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545. 2023-08-07 08:55:02,768 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:55:02,768 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:55:02,768 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,768 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-07 08:55:02,768 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-07 08:55:02,770 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,770 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:55:02,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:55:02,770 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,770 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:55:02,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:55:02,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:55:02,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:02,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:02,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:55:02,772 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-08-07 08:55:02,772 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@590181b1[Count = 0] remaining members to acquire global barrier 2023-08-07 08:55:02,772 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-08-07 08:55:02,772 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,774 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,774 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,774 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,774 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,774 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:55:02,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,775 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-07 08:55:02,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,775 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-07 08:55:02,775 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-07 08:55:02,775 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-07 08:55:02,775 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-07 08:55:02,775 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:55:02,775 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:55:02,775 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-07 08:55:02,775 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:55:02,776 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-07 08:55:02,776 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-07 08:55:02,776 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:02,776 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:02,776 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-07 08:55:02,776 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-07 08:55:02,777 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-07 08:55:02,777 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:02,777 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,777 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-07 08:55:02,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:55:02,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:55:02,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:55:02,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:55:02,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:02,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:02,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:55:02,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:02,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:02,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,34633,1691398396427': 2023-08-07 08:55:02,780 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-08-07 08:55:02,780 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-08-07 08:55:02,780 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:55:02,780 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:55:02,780 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,780 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:55:02,781 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,781 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,781 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:02,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,781 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,781 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,781 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:02,782 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:02,782 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,782 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:02,784 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:02,784 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,784 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:02,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:55:02,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:02,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,782 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:02,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:02,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:55:02,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:02,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:02,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:02,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:55:02,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:02,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:55:02,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:02,790 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:02,790 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,790 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,790 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:02,790 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:02,790 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:02,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,790 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:02,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:02,790 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:02,790 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-08-07 08:55:02,791 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:55:02,790 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:02,790 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:02,791 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:02,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:02,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:02,791 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-08-07 08:55:02,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:02,791 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:02,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:02,791 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:55:02,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:02,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:02,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:02,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:02,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:02,791 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,791 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:02,792 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:55:02,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:02,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:02,792 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:02,792 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,792 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,792 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter because node does not exist (not an error) 2023-08-07 08:55:02,891 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:55:02,892 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:55:02,903 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:55:02,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-08-07 08:55:02,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,904 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-08-07 08:55:02,904 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:02,904 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-08-07 08:55:02,904 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-08-07 08:55:02,904 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,904 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,906 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:02,906 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:02,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:02,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:02,906 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:02,906 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:02,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:02,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:02,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:02,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,906 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,906 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:55:02,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,906 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,907 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,907 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:55:02,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,907 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:55:02,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:55:02,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,907 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,907 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,907 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-07 08:55:02,907 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,907 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-07 08:55:02,907 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:02,907 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-07 08:55:02,907 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-07 08:55:02,907 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,907 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,907 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-07 08:55:02,907 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-07 08:55:02,908 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:02,907 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-07 08:55:02,908 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-07 08:55:02,908 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-07 08:55:02,908 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:02,908 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-07 08:55:02,908 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-07 08:55:02,908 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-07 08:55:02,908 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545. 2023-08-07 08:55:02,908 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-07 08:55:02,908 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:55:02,908 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:55:02,908 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545. started... 2023-08-07 08:55:02,908 INFO [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 42c4fe66166a76787d86f5399d5eb545 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:55:02,909 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,909 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,909 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,909 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,909 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:55:02,909 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:55:02,916 INFO [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.tmp/cf1/1dc240384a77481283048b7c7b75d791 2023-08-07 08:55:02,926 INFO [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.tmp/cf2/d85fb355aff1498480cf3e8df6fdf5b7 2023-08-07 08:55:02,929 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.tmp/cf1/1dc240384a77481283048b7c7b75d791 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/1dc240384a77481283048b7c7b75d791 2023-08-07 08:55:02,932 INFO [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/1dc240384a77481283048b7c7b75d791, entries=10, sequenceid=58, filesize=5.0 K 2023-08-07 08:55:02,933 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.tmp/cf2/d85fb355aff1498480cf3e8df6fdf5b7 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/d85fb355aff1498480cf3e8df6fdf5b7 2023-08-07 08:55:02,936 INFO [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/d85fb355aff1498480cf3e8df6fdf5b7, entries=10, sequenceid=58, filesize=5.0 K 2023-08-07 08:55:02,937 INFO [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 42c4fe66166a76787d86f5399d5eb545 in 29ms, sequenceid=58, compaction requested=true 2023-08-07 08:55:02,937 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 42c4fe66166a76787d86f5399d5eb545: 2023-08-07 08:55:02,937 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545. 2023-08-07 08:55:02,937 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:55:02,937 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:55:02,937 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,937 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-07 08:55:02,937 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-07 08:55:02,940 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,940 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:55:02,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:55:02,940 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,940 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:55:02,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:55:02,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:55:02,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:02,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:02,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:55:02,942 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-08-07 08:55:02,942 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1fc6067b[Count = 0] remaining members to acquire global barrier 2023-08-07 08:55:02,942 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-08-07 08:55:02,942 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,943 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,943 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,943 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,944 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-07 08:55:02,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,944 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-07 08:55:02,944 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-07 08:55:02,944 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-07 08:55:02,944 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:55:02,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,944 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,944 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:55:02,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,944 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:55:02,944 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-07 08:55:02,944 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-07 08:55:02,944 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:55:02,945 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-07 08:55:02,945 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:02,945 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-07 08:55:02,946 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-07 08:55:02,946 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:02,946 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-07 08:55:02,947 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-07 08:55:02,947 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:02,947 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,947 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-07 08:55:02,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:55:02,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:55:02,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:55:02,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:55:02,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:02,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:02,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:55:02,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:02,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:02,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,34633,1691398396427': 2023-08-07 08:55:02,950 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-08-07 08:55:02,950 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-08-07 08:55:02,951 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:55:02,951 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:55:02,951 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,951 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:55:02,952 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,952 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,952 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,952 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:02,952 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,952 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:02,952 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:55:02,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:55:02,952 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:02,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:02,953 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:02,952 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:02,952 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:02,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:55:02,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:02,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:02,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:02,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,953 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,954 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:02,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:55:02,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:02,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:02,955 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:02,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:55:02,955 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,955 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:02,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:02,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:02,958 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:02,958 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,958 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,958 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:02,958 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:02,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:02,958 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:02,958 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:02,958 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-08-07 08:55:02,958 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:55:02,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:02,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:02,958 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:02,958 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:02,958 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-08-07 08:55:02,958 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:55:02,958 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:02,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:02,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:02,958 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:02,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:02,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:02,959 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:02,959 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:02,959 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:02,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:02,959 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:02,959 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:02,959 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:02,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:02,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:02,959 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,059 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:55:03,060 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:55:03,070 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:55:03,072 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-08-07 08:55:03,072 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,073 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-08-07 08:55:03,073 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:03,073 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-08-07 08:55:03,073 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-08-07 08:55:03,073 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,073 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,074 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:03,074 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:03,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:03,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:03,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:03,074 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:03,074 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:03,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:03,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:03,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:03,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,075 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:03,075 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:55:03,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,075 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,075 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:55:03,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,075 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,075 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:55:03,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,076 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,076 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-07 08:55:03,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:55:03,076 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,076 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-07 08:55:03,076 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-07 08:55:03,076 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,076 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-07 08:55:03,076 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:03,076 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:03,076 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-07 08:55:03,076 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-07 08:55:03,076 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-07 08:55:03,076 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,076 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:55:03,076 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-07 08:55:03,076 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:55:03,076 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545. 2023-08-07 08:55:03,076 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545. started... 2023-08-07 08:55:03,076 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-07 08:55:03,076 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,077 INFO [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 42c4fe66166a76787d86f5399d5eb545 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:55:03,077 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-07 08:55:03,077 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:03,077 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-07 08:55:03,077 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-07 08:55:03,077 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-07 08:55:03,078 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,079 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,079 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:55:03,079 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,080 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,080 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:55:03,085 INFO [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.tmp/cf1/caf248b4955e47029493ea10e8d74f0d 2023-08-07 08:55:03,096 INFO [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.tmp/cf2/2ca6789bcc9a491b8b1f7ed5316ba73c 2023-08-07 08:55:03,099 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.tmp/cf1/caf248b4955e47029493ea10e8d74f0d as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/caf248b4955e47029493ea10e8d74f0d 2023-08-07 08:55:03,102 INFO [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/caf248b4955e47029493ea10e8d74f0d, entries=10, sequenceid=71, filesize=5.0 K 2023-08-07 08:55:03,103 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.tmp/cf2/2ca6789bcc9a491b8b1f7ed5316ba73c as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/2ca6789bcc9a491b8b1f7ed5316ba73c 2023-08-07 08:55:03,106 INFO [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/2ca6789bcc9a491b8b1f7ed5316ba73c, entries=10, sequenceid=71, filesize=5.0 K 2023-08-07 08:55:03,107 INFO [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 42c4fe66166a76787d86f5399d5eb545 in 31ms, sequenceid=71, compaction requested=true 2023-08-07 08:55:03,107 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 42c4fe66166a76787d86f5399d5eb545: 2023-08-07 08:55:03,107 DEBUG [rs(jenkins-hbase4.apache.org,34633,1691398396427)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545. 2023-08-07 08:55:03,107 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:55:03,107 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:55:03,107 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:03,107 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-07 08:55:03,107 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-07 08:55:03,109 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,109 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:03,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:03,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:55:03,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:55:03,109 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,109 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:55:03,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:55:03,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:55:03,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:03,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:03,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:03,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:55:03,111 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-08-07 08:55:03,111 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5ca0594d[Count = 0] remaining members to acquire global barrier 2023-08-07 08:55:03,111 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-08-07 08:55:03,111 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,116 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,116 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,116 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,116 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:03,116 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:55:03,116 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-07 08:55:03,116 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-07 08:55:03,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,116 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:55:03,116 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-07 08:55:03,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,116 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-07 08:55:03,116 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-07 08:55:03,116 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-07 08:55:03,116 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:55:03,116 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:55:03,117 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-07 08:55:03,117 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:03,117 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-07 08:55:03,118 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-07 08:55:03,118 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:03,118 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-07 08:55:03,119 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:03,119 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-07 08:55:03,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:03,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:55:03,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:55:03,119 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:03,119 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-07 08:55:03,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:55:03,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:55:03,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:03,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:03,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:03,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:55:03,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:03,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:03,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:03,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,34633,1691398396427': 2023-08-07 08:55:03,122 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-08-07 08:55:03,122 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:55:03,122 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:55:03,122 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,122 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:55:03,122 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-08-07 08:55:03,124 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,124 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:55:03,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:55:03,124 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,124 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:03,124 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:55:03,124 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:03,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,125 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:03,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,124 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:03,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:03,125 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:03,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:03,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:03,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:03,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:03,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:03,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:55:03,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,125 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:03,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:03,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:03,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:03,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:55:03,127 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:03,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,127 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:03,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:03,127 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:03,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:03,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:03,130 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:03,130 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,130 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:03,130 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:03,130 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:03,130 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:03,130 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:03,130 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:03,130 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-08-07 08:55:03,130 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:55:03,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:03,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:03,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:03,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:03,130 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:03,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:03,130 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:03,130 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:03,131 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-08-07 08:55:03,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:03,131 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:55:03,131 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:03,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:03,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:03,131 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:03,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:03,131 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:03,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:03,131 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:03,131 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:03,131 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,131 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-07 08:55:03,231 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:55:03,232 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:55:03,240 DEBUG [Listener at localhost/33561] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:03,240 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(1029): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=42c4fe66166a76787d86f5399d5eb545, daughterA=a28257fc5a176c00d2898c734b9a72b8, daughterB=3bf653b20b293fed30234905df266f34 2023-08-07 08:55:03,241 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=42c4fe66166a76787d86f5399d5eb545, daughterA=a28257fc5a176c00d2898c734b9a72b8, daughterB=3bf653b20b293fed30234905df266f34 2023-08-07 08:55:03,241 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=42c4fe66166a76787d86f5399d5eb545, daughterA=a28257fc5a176c00d2898c734b9a72b8, daughterB=3bf653b20b293fed30234905df266f34 2023-08-07 08:55:03,241 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=42c4fe66166a76787d86f5399d5eb545, daughterA=a28257fc5a176c00d2898c734b9a72b8, daughterB=3bf653b20b293fed30234905df266f34 2023-08-07 08:55:03,243 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=42c4fe66166a76787d86f5399d5eb545, UNASSIGN}] 2023-08-07 08:55:03,244 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=42c4fe66166a76787d86f5399d5eb545, UNASSIGN 2023-08-07 08:55:03,245 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=42c4fe66166a76787d86f5399d5eb545, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:03,245 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1691398503245"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398503245"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398503245"}]},"ts":"1691398503245"} 2023-08-07 08:55:03,246 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure 42c4fe66166a76787d86f5399d5eb545, server=jenkins-hbase4.apache.org,34633,1691398396427}] 2023-08-07 08:55:03,398 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,398 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 42c4fe66166a76787d86f5399d5eb545, disabling compactions & flushes 2023-08-07 08:55:03,398 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545. 2023-08-07 08:55:03,398 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545. 2023-08-07 08:55:03,398 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545. after waiting 0 ms 2023-08-07 08:55:03,398 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545. 2023-08-07 08:55:03,398 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 42c4fe66166a76787d86f5399d5eb545 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-08-07 08:55:03,406 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.tmp/cf1/cf5514b480754e27b952d1577ce1d03d 2023-08-07 08:55:03,418 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.tmp/cf2/f51a84bb914541bebe4d9fa8b9e5f87c 2023-08-07 08:55:03,421 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.tmp/cf1/cf5514b480754e27b952d1577ce1d03d as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/cf5514b480754e27b952d1577ce1d03d 2023-08-07 08:55:03,424 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/cf5514b480754e27b952d1577ce1d03d, entries=8, sequenceid=82, filesize=5.0 K 2023-08-07 08:55:03,425 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.tmp/cf2/f51a84bb914541bebe4d9fa8b9e5f87c as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/f51a84bb914541bebe4d9fa8b9e5f87c 2023-08-07 08:55:03,428 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/f51a84bb914541bebe4d9fa8b9e5f87c, entries=8, sequenceid=82, filesize=5.0 K 2023-08-07 08:55:03,429 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 42c4fe66166a76787d86f5399d5eb545 in 31ms, sequenceid=82, compaction requested=true 2023-08-07 08:55:03,438 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-08-07 08:55:03,439 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-07 08:55:03,440 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545. 2023-08-07 08:55:03,440 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 42c4fe66166a76787d86f5399d5eb545: 2023-08-07 08:55:03,441 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,441 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=42c4fe66166a76787d86f5399d5eb545, regionState=CLOSED 2023-08-07 08:55:03,442 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1691398503441"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398503441"}]},"ts":"1691398503441"} 2023-08-07 08:55:03,444 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=129 2023-08-07 08:55:03,444 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure 42c4fe66166a76787d86f5399d5eb545, server=jenkins-hbase4.apache.org,34633,1691398396427 in 197 msec 2023-08-07 08:55:03,446 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=128 2023-08-07 08:55:03,446 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=42c4fe66166a76787d86f5399d5eb545, UNASSIGN in 201 msec 2023-08-07 08:55:03,454 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=42c4fe66166a76787d86f5399d5eb545, threads=14 2023-08-07 08:55:03,454 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/2ca6789bcc9a491b8b1f7ed5316ba73c for region: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,454 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/615d7da3ff9546eab7f2502413a6b1fa for region: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,454 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/693aae8c23f647cb8cc80322affd4413 for region: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,454 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/6c30eef80f2a43ec8cb3d368f83c2ae9 for region: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,454 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/a658b133e5f74726a921246fd9b66ab3 for region: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,455 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/d85fb355aff1498480cf3e8df6fdf5b7 for region: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,455 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/f51a84bb914541bebe4d9fa8b9e5f87c for region: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,455 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/1cd40e3a18e648c2bb75ad6f4918564d for region: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,455 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/1dc240384a77481283048b7c7b75d791 for region: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,455 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/caf248b4955e47029493ea10e8d74f0d for region: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,455 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/ced9e8182e354b89a6f3d6d205f31c07 for region: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,455 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/cf5514b480754e27b952d1577ce1d03d for region: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,456 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/eb185e0905984e5eb44f41b140d6b93e for region: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,456 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/ee816b081ce843248046865a7e7bd371 for region: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,459 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/2ca6789bcc9a491b8b1f7ed5316ba73c, top=false 2023-08-07 08:55:03,465 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/d85fb355aff1498480cf3e8df6fdf5b7, top=false 2023-08-07 08:55:03,465 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/a658b133e5f74726a921246fd9b66ab3, top=false 2023-08-07 08:55:03,465 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/eb185e0905984e5eb44f41b140d6b93e, top=false 2023-08-07 08:55:03,466 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/f51a84bb914541bebe4d9fa8b9e5f87c, top=false 2023-08-07 08:55:03,469 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/6c30eef80f2a43ec8cb3d368f83c2ae9, top=false 2023-08-07 08:55:03,469 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/cf5514b480754e27b952d1577ce1d03d, top=false 2023-08-07 08:55:03,473 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/caf248b4955e47029493ea10e8d74f0d, top=false 2023-08-07 08:55:03,477 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/ced9e8182e354b89a6f3d6d205f31c07, top=false 2023-08-07 08:55:03,477 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/693aae8c23f647cb8cc80322affd4413, top=false 2023-08-07 08:55:03,477 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/615d7da3ff9546eab7f2502413a6b1fa, top=false 2023-08-07 08:55:03,479 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/1dc240384a77481283048b7c7b75d791, top=false 2023-08-07 08:55:03,483 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.splits/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-2ca6789bcc9a491b8b1f7ed5316ba73c for child: a28257fc5a176c00d2898c734b9a72b8, parent: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,483 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/1cd40e3a18e648c2bb75ad6f4918564d, top=false 2023-08-07 08:55:03,484 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/ee816b081ce843248046865a7e7bd371, top=false 2023-08-07 08:55:03,492 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/2ca6789bcc9a491b8b1f7ed5316ba73c for region: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,495 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.splits/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-cf5514b480754e27b952d1577ce1d03d for child: a28257fc5a176c00d2898c734b9a72b8, parent: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,495 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.splits/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-1dc240384a77481283048b7c7b75d791 for child: a28257fc5a176c00d2898c734b9a72b8, parent: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,496 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.splits/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-d85fb355aff1498480cf3e8df6fdf5b7 for child: a28257fc5a176c00d2898c734b9a72b8, parent: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,499 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/1dc240384a77481283048b7c7b75d791 for region: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,500 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/d85fb355aff1498480cf3e8df6fdf5b7 for region: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,503 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/cf5514b480754e27b952d1577ce1d03d for region: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,506 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.splits/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-693aae8c23f647cb8cc80322affd4413 for child: a28257fc5a176c00d2898c734b9a72b8, parent: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,507 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.splits/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-6c30eef80f2a43ec8cb3d368f83c2ae9 for child: a28257fc5a176c00d2898c734b9a72b8, parent: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,507 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.splits/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-ced9e8182e354b89a6f3d6d205f31c07 for child: a28257fc5a176c00d2898c734b9a72b8, parent: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,511 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.splits/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-caf248b4955e47029493ea10e8d74f0d for child: a28257fc5a176c00d2898c734b9a72b8, parent: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,513 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.splits/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-eb185e0905984e5eb44f41b140d6b93e for child: a28257fc5a176c00d2898c734b9a72b8, parent: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,515 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/6c30eef80f2a43ec8cb3d368f83c2ae9 for region: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,515 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/ced9e8182e354b89a6f3d6d205f31c07 for region: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,516 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.splits/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-1cd40e3a18e648c2bb75ad6f4918564d for child: a28257fc5a176c00d2898c734b9a72b8, parent: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,517 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/caf248b4955e47029493ea10e8d74f0d for region: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,518 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/eb185e0905984e5eb44f41b140d6b93e for region: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,519 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/1cd40e3a18e648c2bb75ad6f4918564d for region: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,523 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.splits/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-ee816b081ce843248046865a7e7bd371 for child: a28257fc5a176c00d2898c734b9a72b8, parent: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,523 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.splits/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-f51a84bb914541bebe4d9fa8b9e5f87c for child: a28257fc5a176c00d2898c734b9a72b8, parent: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,523 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.splits/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-a658b133e5f74726a921246fd9b66ab3 for child: a28257fc5a176c00d2898c734b9a72b8, parent: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,523 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/693aae8c23f647cb8cc80322affd4413 for region: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,523 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/.splits/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-615d7da3ff9546eab7f2502413a6b1fa for child: a28257fc5a176c00d2898c734b9a72b8, parent: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,526 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/f51a84bb914541bebe4d9fa8b9e5f87c for region: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,526 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/ee816b081ce843248046865a7e7bd371 for region: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,527 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/a658b133e5f74726a921246fd9b66ab3 for region: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,527 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/615d7da3ff9546eab7f2502413a6b1fa for region: 42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:03,527 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region 42c4fe66166a76787d86f5399d5eb545 Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-08-07 08:55:03,551 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-08-07 08:55:03,553 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/3bf653b20b293fed30234905df266f34/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-08-07 08:55:03,554 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1691398503554"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1691398503554"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1691398503554"}]},"ts":"1691398503554"} 2023-08-07 08:55:03,554 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1691398503554"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398503554"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398503554"}]},"ts":"1691398503554"} 2023-08-07 08:55:03,554 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1691398503240.3bf653b20b293fed30234905df266f34.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1691398503554"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398503554"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398503554"}]},"ts":"1691398503554"} 2023-08-07 08:55:03,557 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=a28257fc5a176c00d2898c734b9a72b8, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=3bf653b20b293fed30234905df266f34, ASSIGN}] 2023-08-07 08:55:03,558 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=3bf653b20b293fed30234905df266f34, ASSIGN 2023-08-07 08:55:03,558 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=3bf653b20b293fed30234905df266f34, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,34633,1691398396427; forceNewPlan=false, retain=false 2023-08-07 08:55:03,559 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=a28257fc5a176c00d2898c734b9a72b8, ASSIGN 2023-08-07 08:55:03,559 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=a28257fc5a176c00d2898c734b9a72b8, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,34633,1691398396427; forceNewPlan=false, retain=false 2023-08-07 08:55:03,709 INFO [jenkins-hbase4:33357] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-08-07 08:55:03,710 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=3bf653b20b293fed30234905df266f34, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:03,710 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1691398503240.3bf653b20b293fed30234905df266f34.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1691398503710"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398503710"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398503710"}]},"ts":"1691398503710"} 2023-08-07 08:55:03,712 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=132, state=RUNNABLE; OpenRegionProcedure 3bf653b20b293fed30234905df266f34, server=jenkins-hbase4.apache.org,34633,1691398396427}] 2023-08-07 08:55:03,714 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=a28257fc5a176c00d2898c734b9a72b8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:03,714 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1691398503714"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398503714"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398503714"}]},"ts":"1691398503714"} 2023-08-07 08:55:03,715 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=131, state=RUNNABLE; OpenRegionProcedure a28257fc5a176c00d2898c734b9a72b8, server=jenkins-hbase4.apache.org,34633,1691398396427}] 2023-08-07 08:55:03,849 ERROR [BP-1940969354-172.31.14.131-1691398391279 heartbeating to localhost/127.0.0.1:45787] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1940969354-172.31.14.131-1691398391279 (Datanode Uuid 4fd49325-1f52-494b-bfd8-92875de86a16) service to localhost/127.0.0.1:45787 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-08-07 08:55:03,866 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionEmptyDaughter,71,1691398503240.3bf653b20b293fed30234905df266f34. 2023-08-07 08:55:03,866 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 3bf653b20b293fed30234905df266f34, NAME => 'testSplitTableRegionEmptyDaughter,71,1691398503240.3bf653b20b293fed30234905df266f34.', STARTKEY => '71', ENDKEY => ''} 2023-08-07 08:55:03,866 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-07 08:55:03,866 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 3bf653b20b293fed30234905df266f34 2023-08-07 08:55:03,867 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,71,1691398503240.3bf653b20b293fed30234905df266f34.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:55:03,867 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 3bf653b20b293fed30234905df266f34 2023-08-07 08:55:03,867 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 3bf653b20b293fed30234905df266f34 2023-08-07 08:55:03,868 INFO [StoreOpener-3bf653b20b293fed30234905df266f34-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 3bf653b20b293fed30234905df266f34 2023-08-07 08:55:03,869 DEBUG [StoreOpener-3bf653b20b293fed30234905df266f34-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/3bf653b20b293fed30234905df266f34/cf1 2023-08-07 08:55:03,869 DEBUG [StoreOpener-3bf653b20b293fed30234905df266f34-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/3bf653b20b293fed30234905df266f34/cf1 2023-08-07 08:55:03,869 INFO [StoreOpener-3bf653b20b293fed30234905df266f34-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3bf653b20b293fed30234905df266f34 columnFamilyName cf1 2023-08-07 08:55:03,870 INFO [StoreOpener-3bf653b20b293fed30234905df266f34-1] regionserver.HStore(310): Store=3bf653b20b293fed30234905df266f34/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:55:03,870 INFO [StoreOpener-3bf653b20b293fed30234905df266f34-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 3bf653b20b293fed30234905df266f34 2023-08-07 08:55:03,871 DEBUG [StoreOpener-3bf653b20b293fed30234905df266f34-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/3bf653b20b293fed30234905df266f34/cf2 2023-08-07 08:55:03,871 DEBUG [StoreOpener-3bf653b20b293fed30234905df266f34-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/3bf653b20b293fed30234905df266f34/cf2 2023-08-07 08:55:03,871 INFO [StoreOpener-3bf653b20b293fed30234905df266f34-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3bf653b20b293fed30234905df266f34 columnFamilyName cf2 2023-08-07 08:55:03,872 INFO [StoreOpener-3bf653b20b293fed30234905df266f34-1] regionserver.HStore(310): Store=3bf653b20b293fed30234905df266f34/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:55:03,873 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/3bf653b20b293fed30234905df266f34 2023-08-07 08:55:03,874 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/3bf653b20b293fed30234905df266f34 2023-08-07 08:55:03,875 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-07 08:55:03,876 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 3bf653b20b293fed30234905df266f34 2023-08-07 08:55:03,877 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 3bf653b20b293fed30234905df266f34; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68639798, jitterRate=0.022812694311141968}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-07 08:55:03,877 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 3bf653b20b293fed30234905df266f34: 2023-08-07 08:55:03,877 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1691398503240.3bf653b20b293fed30234905df266f34., pid=133, masterSystemTime=1691398503863 2023-08-07 08:55:03,878 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1691398503240.3bf653b20b293fed30234905df266f34. 2023-08-07 08:55:03,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionEmptyDaughter,71,1691398503240.3bf653b20b293fed30234905df266f34. 2023-08-07 08:55:03,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8. 2023-08-07 08:55:03,878 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a28257fc5a176c00d2898c734b9a72b8, NAME => 'testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8.', STARTKEY => '', ENDKEY => '71'} 2023-08-07 08:55:03,879 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-07 08:55:03,879 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=3bf653b20b293fed30234905df266f34, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:03,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter a28257fc5a176c00d2898c734b9a72b8 2023-08-07 08:55:03,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:55:03,879 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1691398503240.3bf653b20b293fed30234905df266f34.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1691398503879"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691398503879"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691398503879"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398503879"}]},"ts":"1691398503879"} 2023-08-07 08:55:03,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a28257fc5a176c00d2898c734b9a72b8 2023-08-07 08:55:03,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a28257fc5a176c00d2898c734b9a72b8 2023-08-07 08:55:03,880 INFO [StoreOpener-a28257fc5a176c00d2898c734b9a72b8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a28257fc5a176c00d2898c734b9a72b8 2023-08-07 08:55:03,881 DEBUG [StoreOpener-a28257fc5a176c00d2898c734b9a72b8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1 2023-08-07 08:55:03,881 DEBUG [StoreOpener-a28257fc5a176c00d2898c734b9a72b8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1 2023-08-07 08:55:03,882 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=132 2023-08-07 08:55:03,882 INFO [StoreOpener-a28257fc5a176c00d2898c734b9a72b8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a28257fc5a176c00d2898c734b9a72b8 columnFamilyName cf1 2023-08-07 08:55:03,882 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=132, state=SUCCESS; OpenRegionProcedure 3bf653b20b293fed30234905df266f34, server=jenkins-hbase4.apache.org,34633,1691398396427 in 168 msec 2023-08-07 08:55:03,883 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=3bf653b20b293fed30234905df266f34, ASSIGN in 325 msec 2023-08-07 08:55:03,887 DEBUG [StoreOpener-a28257fc5a176c00d2898c734b9a72b8-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-1cd40e3a18e648c2bb75ad6f4918564d 2023-08-07 08:55:03,890 DEBUG [StoreOpener-a28257fc5a176c00d2898c734b9a72b8-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-1dc240384a77481283048b7c7b75d791 2023-08-07 08:55:03,893 DEBUG [StoreOpener-a28257fc5a176c00d2898c734b9a72b8-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-caf248b4955e47029493ea10e8d74f0d 2023-08-07 08:55:03,896 DEBUG [StoreOpener-a28257fc5a176c00d2898c734b9a72b8-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-ced9e8182e354b89a6f3d6d205f31c07 2023-08-07 08:55:03,899 DEBUG [StoreOpener-a28257fc5a176c00d2898c734b9a72b8-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-cf5514b480754e27b952d1577ce1d03d 2023-08-07 08:55:03,901 DEBUG [StoreOpener-a28257fc5a176c00d2898c734b9a72b8-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-eb185e0905984e5eb44f41b140d6b93e 2023-08-07 08:55:03,904 DEBUG [StoreOpener-a28257fc5a176c00d2898c734b9a72b8-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-ee816b081ce843248046865a7e7bd371 2023-08-07 08:55:03,904 INFO [StoreOpener-a28257fc5a176c00d2898c734b9a72b8-1] regionserver.HStore(310): Store=a28257fc5a176c00d2898c734b9a72b8/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:55:03,905 INFO [StoreOpener-a28257fc5a176c00d2898c734b9a72b8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region a28257fc5a176c00d2898c734b9a72b8 2023-08-07 08:55:03,905 DEBUG [StoreOpener-a28257fc5a176c00d2898c734b9a72b8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2 2023-08-07 08:55:03,905 DEBUG [StoreOpener-a28257fc5a176c00d2898c734b9a72b8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2 2023-08-07 08:55:03,906 INFO [StoreOpener-a28257fc5a176c00d2898c734b9a72b8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a28257fc5a176c00d2898c734b9a72b8 columnFamilyName cf2 2023-08-07 08:55:03,911 DEBUG [StoreOpener-a28257fc5a176c00d2898c734b9a72b8-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-2ca6789bcc9a491b8b1f7ed5316ba73c 2023-08-07 08:55:03,913 DEBUG [StoreOpener-a28257fc5a176c00d2898c734b9a72b8-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-615d7da3ff9546eab7f2502413a6b1fa 2023-08-07 08:55:03,917 DEBUG [StoreOpener-a28257fc5a176c00d2898c734b9a72b8-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-693aae8c23f647cb8cc80322affd4413 2023-08-07 08:55:03,920 DEBUG [StoreOpener-a28257fc5a176c00d2898c734b9a72b8-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-6c30eef80f2a43ec8cb3d368f83c2ae9 2023-08-07 08:55:03,923 DEBUG [StoreOpener-a28257fc5a176c00d2898c734b9a72b8-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-a658b133e5f74726a921246fd9b66ab3 2023-08-07 08:55:03,926 DEBUG [StoreOpener-a28257fc5a176c00d2898c734b9a72b8-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-d85fb355aff1498480cf3e8df6fdf5b7 2023-08-07 08:55:03,928 DEBUG [StoreOpener-a28257fc5a176c00d2898c734b9a72b8-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-f51a84bb914541bebe4d9fa8b9e5f87c 2023-08-07 08:55:03,929 INFO [StoreOpener-a28257fc5a176c00d2898c734b9a72b8-1] regionserver.HStore(310): Store=a28257fc5a176c00d2898c734b9a72b8/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:55:03,929 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8 2023-08-07 08:55:03,930 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8 2023-08-07 08:55:03,932 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-07 08:55:03,933 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a28257fc5a176c00d2898c734b9a72b8 2023-08-07 08:55:03,933 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a28257fc5a176c00d2898c734b9a72b8; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69802430, jitterRate=0.040137261152267456}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-07 08:55:03,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a28257fc5a176c00d2898c734b9a72b8: 2023-08-07 08:55:03,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8., pid=134, masterSystemTime=1691398503863 2023-08-07 08:55:03,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:55:03,935 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-07 08:55:03,936 DEBUG [RS:1;jenkins-hbase4:34633-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-07 08:55:03,936 DEBUG [RS:1;jenkins-hbase4:34633-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-07 08:55:03,936 DEBUG [RS:1;jenkins-hbase4:34633-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-08-07 08:55:03,936 DEBUG [RS:1;jenkins-hbase4:34633-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-08-07 08:55:03,936 DEBUG [RS:1;jenkins-hbase4:34633-shortCompactions-0] regionserver.HStore(1912): a28257fc5a176c00d2898c734b9a72b8/cf1 is initiating minor compaction (all files) 2023-08-07 08:55:03,936 DEBUG [RS:1;jenkins-hbase4:34633-longCompactions-0] regionserver.HStore(1912): a28257fc5a176c00d2898c734b9a72b8/cf2 is initiating minor compaction (all files) 2023-08-07 08:55:03,936 INFO [RS:1;jenkins-hbase4:34633-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of a28257fc5a176c00d2898c734b9a72b8/cf1 in testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8. 2023-08-07 08:55:03,936 INFO [RS:1;jenkins-hbase4:34633-longCompactions-0] regionserver.HRegion(2259): Starting compaction of a28257fc5a176c00d2898c734b9a72b8/cf2 in testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8. 2023-08-07 08:55:03,936 INFO [RS:1;jenkins-hbase4:34633-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-1cd40e3a18e648c2bb75ad6f4918564d, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-eb185e0905984e5eb44f41b140d6b93e, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-ee816b081ce843248046865a7e7bd371, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-ced9e8182e354b89a6f3d6d205f31c07, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-1dc240384a77481283048b7c7b75d791, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-caf248b4955e47029493ea10e8d74f0d, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-cf5514b480754e27b952d1577ce1d03d] into tmpdir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/.tmp, totalSize=34.9 K 2023-08-07 08:55:03,936 INFO [RS:1;jenkins-hbase4:34633-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-6c30eef80f2a43ec8cb3d368f83c2ae9, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-693aae8c23f647cb8cc80322affd4413, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-a658b133e5f74726a921246fd9b66ab3, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-615d7da3ff9546eab7f2502413a6b1fa, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-d85fb355aff1498480cf3e8df6fdf5b7, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-2ca6789bcc9a491b8b1f7ed5316ba73c, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-f51a84bb914541bebe4d9fa8b9e5f87c] into tmpdir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/.tmp, totalSize=34.9 K 2023-08-07 08:55:03,937 DEBUG [RS:1;jenkins-hbase4:34633-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-1cd40e3a18e648c2bb75ad6f4918564d, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1691398501789 2023-08-07 08:55:03,937 DEBUG [RS:1;jenkins-hbase4:34633-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-6c30eef80f2a43ec8cb3d368f83c2ae9, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1691398501789 2023-08-07 08:55:03,937 DEBUG [RS:1;jenkins-hbase4:34633-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-eb185e0905984e5eb44f41b140d6b93e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1691398501964 2023-08-07 08:55:03,937 DEBUG [RS:1;jenkins-hbase4:34633-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-693aae8c23f647cb8cc80322affd4413, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1691398501964 2023-08-07 08:55:03,938 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8. 2023-08-07 08:55:03,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8. 2023-08-07 08:55:03,938 DEBUG [RS:1;jenkins-hbase4:34633-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-ee816b081ce843248046865a7e7bd371, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1691398502538 2023-08-07 08:55:03,938 DEBUG [RS:1;jenkins-hbase4:34633-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-a658b133e5f74726a921246fd9b66ab3, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1691398502538 2023-08-07 08:55:03,938 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=a28257fc5a176c00d2898c734b9a72b8, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:03,938 DEBUG [RS:1;jenkins-hbase4:34633-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-ced9e8182e354b89a6f3d6d205f31c07, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1691398502713 2023-08-07 08:55:03,938 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1691398503938"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691398503938"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691398503938"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398503938"}]},"ts":"1691398503938"} 2023-08-07 08:55:03,939 DEBUG [RS:1;jenkins-hbase4:34633-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-615d7da3ff9546eab7f2502413a6b1fa, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1691398502713 2023-08-07 08:55:03,939 DEBUG [RS:1;jenkins-hbase4:34633-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-1dc240384a77481283048b7c7b75d791, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1691398502893 2023-08-07 08:55:03,939 DEBUG [RS:1;jenkins-hbase4:34633-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-d85fb355aff1498480cf3e8df6fdf5b7, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1691398502893 2023-08-07 08:55:03,940 DEBUG [RS:1;jenkins-hbase4:34633-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-caf248b4955e47029493ea10e8d74f0d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1691398503061 2023-08-07 08:55:03,940 DEBUG [RS:1;jenkins-hbase4:34633-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-2ca6789bcc9a491b8b1f7ed5316ba73c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1691398503061 2023-08-07 08:55:03,940 DEBUG [RS:1;jenkins-hbase4:34633-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-cf5514b480754e27b952d1577ce1d03d, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1691398503233 2023-08-07 08:55:03,940 DEBUG [RS:1;jenkins-hbase4:34633-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-f51a84bb914541bebe4d9fa8b9e5f87c, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1691398503233 2023-08-07 08:55:03,942 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=131 2023-08-07 08:55:03,942 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=131, state=SUCCESS; OpenRegionProcedure a28257fc5a176c00d2898c734b9a72b8, server=jenkins-hbase4.apache.org,34633,1691398396427 in 225 msec 2023-08-07 08:55:03,945 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=131, resume processing ppid=128 2023-08-07 08:55:03,945 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=a28257fc5a176c00d2898c734b9a72b8, ASSIGN in 385 msec 2023-08-07 08:55:03,946 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=42c4fe66166a76787d86f5399d5eb545, daughterA=a28257fc5a176c00d2898c734b9a72b8, daughterB=3bf653b20b293fed30234905df266f34 in 705 msec 2023-08-07 08:55:03,967 INFO [RS:1;jenkins-hbase4:34633-longCompactions-0] throttle.PressureAwareThroughputController(145): a28257fc5a176c00d2898c734b9a72b8#cf2#compaction#124 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-07 08:55:03,971 INFO [RS:1;jenkins-hbase4:34633-shortCompactions-0] throttle.PressureAwareThroughputController(145): a28257fc5a176c00d2898c734b9a72b8#cf1#compaction#125 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-07 08:55:03,981 DEBUG [RS:1;jenkins-hbase4:34633-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/.tmp/cf2/568f98099c0e48999d47679e04aa07c2 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/568f98099c0e48999d47679e04aa07c2 2023-08-07 08:55:03,981 DEBUG [RS:1;jenkins-hbase4:34633-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/.tmp/cf1/73a0f755462d40c69180cf39238b5fb4 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/73a0f755462d40c69180cf39238b5fb4 2023-08-07 08:55:03,985 INFO [RS:1;jenkins-hbase4:34633-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a28257fc5a176c00d2898c734b9a72b8/cf2 of a28257fc5a176c00d2898c734b9a72b8 into 568f98099c0e48999d47679e04aa07c2(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 08:55:03,985 INFO [RS:1;jenkins-hbase4:34633-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a28257fc5a176c00d2898c734b9a72b8/cf1 of a28257fc5a176c00d2898c734b9a72b8 into 73a0f755462d40c69180cf39238b5fb4(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 08:55:03,985 DEBUG [RS:1;jenkins-hbase4:34633-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for a28257fc5a176c00d2898c734b9a72b8: 2023-08-07 08:55:03,985 DEBUG [RS:1;jenkins-hbase4:34633-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for a28257fc5a176c00d2898c734b9a72b8: 2023-08-07 08:55:03,986 INFO [RS:1;jenkins-hbase4:34633-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8., storeName=a28257fc5a176c00d2898c734b9a72b8/cf2, priority=9, startTime=1691398503935; duration=0sec 2023-08-07 08:55:03,986 INFO [RS:1;jenkins-hbase4:34633-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8., storeName=a28257fc5a176c00d2898c734b9a72b8/cf1, priority=9, startTime=1691398503934; duration=0sec 2023-08-07 08:55:03,987 DEBUG [RS:1;jenkins-hbase4:34633-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:55:03,987 DEBUG [RS:1;jenkins-hbase4:34633-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:55:03,991 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34633] ipc.CallRunner(144): callId: 989 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:51646 deadline: 1691398563991, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545. is not online on jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:04,101 DEBUG [Listener at localhost/33561] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-08-07 08:55:04,104 WARN [Listener at localhost/33561] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-07 08:55:04,105 INFO [Listener at localhost/33561] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-08-07 08:55:04,106 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionEmptyDaughter 2023-08-07 08:55:04,106 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure2.ProcedureExecutor(1029): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-08-07 08:55:04,109 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=135 2023-08-07 08:55:04,109 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398504109"}]},"ts":"1691398504109"} 2023-08-07 08:55:04,110 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-08-07 08:55:04,111 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-08-07 08:55:04,112 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=a28257fc5a176c00d2898c734b9a72b8, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=3bf653b20b293fed30234905df266f34, UNASSIGN}] 2023-08-07 08:55:04,112 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=3bf653b20b293fed30234905df266f34, UNASSIGN 2023-08-07 08:55:04,113 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=3bf653b20b293fed30234905df266f34, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:04,113 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1691398503240.3bf653b20b293fed30234905df266f34.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1691398504113"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398504113"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398504113"}]},"ts":"1691398504113"} 2023-08-07 08:55:04,114 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure 3bf653b20b293fed30234905df266f34, server=jenkins-hbase4.apache.org,34633,1691398396427}] 2023-08-07 08:55:04,116 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=a28257fc5a176c00d2898c734b9a72b8, UNASSIGN 2023-08-07 08:55:04,117 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=a28257fc5a176c00d2898c734b9a72b8, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:04,117 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1691398504117"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398504117"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398504117"}]},"ts":"1691398504117"} 2023-08-07 08:55:04,118 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure a28257fc5a176c00d2898c734b9a72b8, server=jenkins-hbase4.apache.org,34633,1691398396427}] 2023-08-07 08:55:04,210 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=135 2023-08-07 08:55:04,266 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 3bf653b20b293fed30234905df266f34 2023-08-07 08:55:04,267 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 3bf653b20b293fed30234905df266f34, disabling compactions & flushes 2023-08-07 08:55:04,268 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,71,1691398503240.3bf653b20b293fed30234905df266f34. 2023-08-07 08:55:04,268 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1691398503240.3bf653b20b293fed30234905df266f34. 2023-08-07 08:55:04,268 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1691398503240.3bf653b20b293fed30234905df266f34. after waiting 0 ms 2023-08-07 08:55:04,268 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1691398503240.3bf653b20b293fed30234905df266f34. 2023-08-07 08:55:04,271 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/3bf653b20b293fed30234905df266f34/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-08-07 08:55:04,271 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-07 08:55:04,271 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,71,1691398503240.3bf653b20b293fed30234905df266f34. 2023-08-07 08:55:04,271 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 3bf653b20b293fed30234905df266f34: 2023-08-07 08:55:04,273 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 3bf653b20b293fed30234905df266f34 2023-08-07 08:55:04,273 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close a28257fc5a176c00d2898c734b9a72b8 2023-08-07 08:55:04,273 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a28257fc5a176c00d2898c734b9a72b8, disabling compactions & flushes 2023-08-07 08:55:04,273 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8. 2023-08-07 08:55:04,273 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8. 2023-08-07 08:55:04,273 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8. after waiting 0 ms 2023-08-07 08:55:04,273 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8. 2023-08-07 08:55:04,273 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=3bf653b20b293fed30234905df266f34, regionState=CLOSED 2023-08-07 08:55:04,273 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1691398503240.3bf653b20b293fed30234905df266f34.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1691398504273"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398504273"}]},"ts":"1691398504273"} 2023-08-07 08:55:04,273 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-1cd40e3a18e648c2bb75ad6f4918564d, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-eb185e0905984e5eb44f41b140d6b93e, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-ee816b081ce843248046865a7e7bd371, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-ced9e8182e354b89a6f3d6d205f31c07, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-1dc240384a77481283048b7c7b75d791, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-caf248b4955e47029493ea10e8d74f0d, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-cf5514b480754e27b952d1577ce1d03d] to archive 2023-08-07 08:55:04,274 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-07 08:55:04,276 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=137 2023-08-07 08:55:04,276 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure 3bf653b20b293fed30234905df266f34, server=jenkins-hbase4.apache.org,34633,1691398396427 in 160 msec 2023-08-07 08:55:04,277 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=3bf653b20b293fed30234905df266f34, UNASSIGN in 164 msec 2023-08-07 08:55:04,283 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-1cd40e3a18e648c2bb75ad6f4918564d to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-1cd40e3a18e648c2bb75ad6f4918564d 2023-08-07 08:55:04,285 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-eb185e0905984e5eb44f41b140d6b93e to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-eb185e0905984e5eb44f41b140d6b93e 2023-08-07 08:55:04,287 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-ee816b081ce843248046865a7e7bd371 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-ee816b081ce843248046865a7e7bd371 2023-08-07 08:55:04,288 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-ced9e8182e354b89a6f3d6d205f31c07 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-ced9e8182e354b89a6f3d6d205f31c07 2023-08-07 08:55:04,288 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-1dc240384a77481283048b7c7b75d791 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-1dc240384a77481283048b7c7b75d791 2023-08-07 08:55:04,289 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-caf248b4955e47029493ea10e8d74f0d to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-caf248b4955e47029493ea10e8d74f0d 2023-08-07 08:55:04,290 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-cf5514b480754e27b952d1577ce1d03d to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-cf5514b480754e27b952d1577ce1d03d 2023-08-07 08:55:04,292 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-6c30eef80f2a43ec8cb3d368f83c2ae9, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-693aae8c23f647cb8cc80322affd4413, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-a658b133e5f74726a921246fd9b66ab3, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-615d7da3ff9546eab7f2502413a6b1fa, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-d85fb355aff1498480cf3e8df6fdf5b7, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-2ca6789bcc9a491b8b1f7ed5316ba73c, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-f51a84bb914541bebe4d9fa8b9e5f87c] to archive 2023-08-07 08:55:04,293 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-07 08:55:04,294 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-6c30eef80f2a43ec8cb3d368f83c2ae9 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-6c30eef80f2a43ec8cb3d368f83c2ae9 2023-08-07 08:55:04,295 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-693aae8c23f647cb8cc80322affd4413 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-693aae8c23f647cb8cc80322affd4413 2023-08-07 08:55:04,296 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-a658b133e5f74726a921246fd9b66ab3 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-a658b133e5f74726a921246fd9b66ab3 2023-08-07 08:55:04,297 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-615d7da3ff9546eab7f2502413a6b1fa to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-615d7da3ff9546eab7f2502413a6b1fa 2023-08-07 08:55:04,298 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-d85fb355aff1498480cf3e8df6fdf5b7 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-d85fb355aff1498480cf3e8df6fdf5b7 2023-08-07 08:55:04,300 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-2ca6789bcc9a491b8b1f7ed5316ba73c to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-2ca6789bcc9a491b8b1f7ed5316ba73c 2023-08-07 08:55:04,301 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-f51a84bb914541bebe4d9fa8b9e5f87c to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/testSplitTableRegionEmptyDaughter=42c4fe66166a76787d86f5399d5eb545-f51a84bb914541bebe4d9fa8b9e5f87c 2023-08-07 08:55:04,304 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-08-07 08:55:04,305 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-07 08:55:04,305 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8. 2023-08-07 08:55:04,305 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a28257fc5a176c00d2898c734b9a72b8: 2023-08-07 08:55:04,306 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed a28257fc5a176c00d2898c734b9a72b8 2023-08-07 08:55:04,306 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=a28257fc5a176c00d2898c734b9a72b8, regionState=CLOSED 2023-08-07 08:55:04,307 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1691398504306"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398504306"}]},"ts":"1691398504306"} 2023-08-07 08:55:04,309 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=139, resume processing ppid=136 2023-08-07 08:55:04,309 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure a28257fc5a176c00d2898c734b9a72b8, server=jenkins-hbase4.apache.org,34633,1691398396427 in 189 msec 2023-08-07 08:55:04,310 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=136, resume processing ppid=135 2023-08-07 08:55:04,310 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=a28257fc5a176c00d2898c734b9a72b8, UNASSIGN in 197 msec 2023-08-07 08:55:04,311 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398504311"}]},"ts":"1691398504311"} 2023-08-07 08:55:04,312 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-08-07 08:55:04,314 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-08-07 08:55:04,315 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 208 msec 2023-08-07 08:55:04,411 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=135 2023-08-07 08:55:04,411 INFO [Listener at localhost/33561] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-08-07 08:55:04,412 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionEmptyDaughter 2023-08-07 08:55:04,413 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure2.ProcedureExecutor(1029): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-08-07 08:55:04,415 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-08-07 08:55:04,415 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-08-07 08:55:04,415 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=140 2023-08-07 08:55:04,419 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:04,419 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/3bf653b20b293fed30234905df266f34 2023-08-07 08:55:04,419 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8 2023-08-07 08:55:04,421 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/3bf653b20b293fed30234905df266f34/cf1, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/3bf653b20b293fed30234905df266f34/cf2, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/3bf653b20b293fed30234905df266f34/recovered.edits] 2023-08-07 08:55:04,421 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/recovered.edits] 2023-08-07 08:55:04,421 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/recovered.edits] 2023-08-07 08:55:04,424 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/1cd40e3a18e648c2bb75ad6f4918564d to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/1cd40e3a18e648c2bb75ad6f4918564d 2023-08-07 08:55:04,425 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/73a0f755462d40c69180cf39238b5fb4 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf1/73a0f755462d40c69180cf39238b5fb4 2023-08-07 08:55:04,426 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/1dc240384a77481283048b7c7b75d791 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/1dc240384a77481283048b7c7b75d791 2023-08-07 08:55:04,427 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/3bf653b20b293fed30234905df266f34/recovered.edits/88.seqid to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionEmptyDaughter/3bf653b20b293fed30234905df266f34/recovered.edits/88.seqid 2023-08-07 08:55:04,427 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/caf248b4955e47029493ea10e8d74f0d to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/caf248b4955e47029493ea10e8d74f0d 2023-08-07 08:55:04,427 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/568f98099c0e48999d47679e04aa07c2 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/cf2/568f98099c0e48999d47679e04aa07c2 2023-08-07 08:55:04,428 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/3bf653b20b293fed30234905df266f34 2023-08-07 08:55:04,428 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/ced9e8182e354b89a6f3d6d205f31c07 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/ced9e8182e354b89a6f3d6d205f31c07 2023-08-07 08:55:04,430 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/cf5514b480754e27b952d1577ce1d03d to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/cf5514b480754e27b952d1577ce1d03d 2023-08-07 08:55:04,430 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/recovered.edits/90.seqid to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8/recovered.edits/90.seqid 2023-08-07 08:55:04,430 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/a28257fc5a176c00d2898c734b9a72b8 2023-08-07 08:55:04,431 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/eb185e0905984e5eb44f41b140d6b93e to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/eb185e0905984e5eb44f41b140d6b93e 2023-08-07 08:55:04,432 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/ee816b081ce843248046865a7e7bd371 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf1/ee816b081ce843248046865a7e7bd371 2023-08-07 08:55:04,434 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/2ca6789bcc9a491b8b1f7ed5316ba73c to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/2ca6789bcc9a491b8b1f7ed5316ba73c 2023-08-07 08:55:04,435 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/615d7da3ff9546eab7f2502413a6b1fa to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/615d7da3ff9546eab7f2502413a6b1fa 2023-08-07 08:55:04,436 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/693aae8c23f647cb8cc80322affd4413 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/693aae8c23f647cb8cc80322affd4413 2023-08-07 08:55:04,437 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/6c30eef80f2a43ec8cb3d368f83c2ae9 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/6c30eef80f2a43ec8cb3d368f83c2ae9 2023-08-07 08:55:04,439 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/a658b133e5f74726a921246fd9b66ab3 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/a658b133e5f74726a921246fd9b66ab3 2023-08-07 08:55:04,440 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/d85fb355aff1498480cf3e8df6fdf5b7 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/d85fb355aff1498480cf3e8df6fdf5b7 2023-08-07 08:55:04,441 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/f51a84bb914541bebe4d9fa8b9e5f87c to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/cf2/f51a84bb914541bebe4d9fa8b9e5f87c 2023-08-07 08:55:04,443 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/recovered.edits/85.seqid to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545/recovered.edits/85.seqid 2023-08-07 08:55:04,444 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegionEmptyDaughter/42c4fe66166a76787d86f5399d5eb545 2023-08-07 08:55:04,444 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-08-07 08:55:04,446 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-08-07 08:55:04,448 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-08-07 08:55:04,450 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-08-07 08:55:04,451 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-08-07 08:55:04,451 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-08-07 08:55:04,451 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691398504451"}]},"ts":"9223372036854775807"} 2023-08-07 08:55:04,451 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691398504451"}]},"ts":"9223372036854775807"} 2023-08-07 08:55:04,451 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1691398503240.3bf653b20b293fed30234905df266f34.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691398504451"}]},"ts":"9223372036854775807"} 2023-08-07 08:55:04,453 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-08-07 08:55:04,453 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 42c4fe66166a76787d86f5399d5eb545, NAME => 'testSplitTableRegionEmptyDaughter,,1691398500781.42c4fe66166a76787d86f5399d5eb545.', STARTKEY => '', ENDKEY => ''}, {ENCODED => a28257fc5a176c00d2898c734b9a72b8, NAME => 'testSplitTableRegionEmptyDaughter,,1691398503240.a28257fc5a176c00d2898c734b9a72b8.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => 3bf653b20b293fed30234905df266f34, NAME => 'testSplitTableRegionEmptyDaughter,71,1691398503240.3bf653b20b293fed30234905df266f34.', STARTKEY => '71', ENDKEY => ''}] 2023-08-07 08:55:04,453 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-08-07 08:55:04,453 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1691398504453"}]},"ts":"9223372036854775807"} 2023-08-07 08:55:04,454 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-08-07 08:55:04,457 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-08-07 08:55:04,458 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 45 msec 2023-08-07 08:55:04,516 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=140 2023-08-07 08:55:04,517 INFO [Listener at localhost/33561] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-08-07 08:55:04,531 INFO [Listener at localhost/33561] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=377 (was 374) - Thread LEAK? -, OpenFileDescriptor=838 (was 836) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=202 (was 194) - SystemLoadAverage LEAK? -, ProcessCount=175 (was 175), AvailableMemoryMB=1805 (was 1805) 2023-08-07 08:55:04,546 INFO [Listener at localhost/33561] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=377, OpenFileDescriptor=838, MaxFileDescriptor=60000, SystemLoadAverage=202, ProcessCount=175, AvailableMemoryMB=1805 2023-08-07 08:55:04,546 WARN [Listener at localhost/33561] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-07 08:55:04,548 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-08-07 08:55:04,548 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-08-07 08:55:04,549 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(1029): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-08-07 08:55:04,551 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-08-07 08:55:04,552 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-07 08:55:04,553 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:04,553 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c empty. 2023-08-07 08:55:04,554 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:04,554 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-08-07 08:55:04,566 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-08-07 08:55:04,567 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7675): creating {ENCODED => ee46c734c4c3f049b054ec6f34f49d5c, NAME => 'testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp 2023-08-07 08:55:04,574 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:55:04,574 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1604): Closing ee46c734c4c3f049b054ec6f34f49d5c, disabling compactions & flushes 2023-08-07 08:55:04,574 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c. 2023-08-07 08:55:04,574 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c. 2023-08-07 08:55:04,574 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c. after waiting 0 ms 2023-08-07 08:55:04,574 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c. 2023-08-07 08:55:04,574 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c. 2023-08-07 08:55:04,574 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1558): Region close journal for ee46c734c4c3f049b054ec6f34f49d5c: 2023-08-07 08:55:04,576 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-08-07 08:55:04,577 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1691398504577"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398504577"}]},"ts":"1691398504577"} 2023-08-07 08:55:04,578 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-07 08:55:04,579 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-07 08:55:04,579 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398504579"}]},"ts":"1691398504579"} 2023-08-07 08:55:04,580 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-08-07 08:55:04,582 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-07 08:55:04,583 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-07 08:55:04,583 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-07 08:55:04,583 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-07 08:55:04,583 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-07 08:55:04,584 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=ee46c734c4c3f049b054ec6f34f49d5c, ASSIGN}] 2023-08-07 08:55:04,585 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=ee46c734c4c3f049b054ec6f34f49d5c, ASSIGN 2023-08-07 08:55:04,585 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=ee46c734c4c3f049b054ec6f34f49d5c, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34559,1691398396467; forceNewPlan=false, retain=false 2023-08-07 08:55:04,735 INFO [jenkins-hbase4:33357] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-07 08:55:04,737 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=ee46c734c4c3f049b054ec6f34f49d5c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:04,737 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1691398504736"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398504736"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398504736"}]},"ts":"1691398504736"} 2023-08-07 08:55:04,738 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure ee46c734c4c3f049b054ec6f34f49d5c, server=jenkins-hbase4.apache.org,34559,1691398396467}] 2023-08-07 08:55:04,892 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c. 2023-08-07 08:55:04,892 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ee46c734c4c3f049b054ec6f34f49d5c, NAME => 'testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c.', STARTKEY => '', ENDKEY => ''} 2023-08-07 08:55:04,892 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-07 08:55:04,892 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:04,892 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:55:04,893 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:04,893 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:04,894 INFO [StoreOpener-ee46c734c4c3f049b054ec6f34f49d5c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:04,895 DEBUG [StoreOpener-ee46c734c4c3f049b054ec6f34f49d5c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1 2023-08-07 08:55:04,895 DEBUG [StoreOpener-ee46c734c4c3f049b054ec6f34f49d5c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1 2023-08-07 08:55:04,895 INFO [StoreOpener-ee46c734c4c3f049b054ec6f34f49d5c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ee46c734c4c3f049b054ec6f34f49d5c columnFamilyName cf1 2023-08-07 08:55:04,896 INFO [StoreOpener-ee46c734c4c3f049b054ec6f34f49d5c-1] regionserver.HStore(310): Store=ee46c734c4c3f049b054ec6f34f49d5c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:55:04,896 INFO [StoreOpener-ee46c734c4c3f049b054ec6f34f49d5c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:04,897 DEBUG [StoreOpener-ee46c734c4c3f049b054ec6f34f49d5c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2 2023-08-07 08:55:04,897 DEBUG [StoreOpener-ee46c734c4c3f049b054ec6f34f49d5c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2 2023-08-07 08:55:04,897 INFO [StoreOpener-ee46c734c4c3f049b054ec6f34f49d5c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ee46c734c4c3f049b054ec6f34f49d5c columnFamilyName cf2 2023-08-07 08:55:04,898 INFO [StoreOpener-ee46c734c4c3f049b054ec6f34f49d5c-1] regionserver.HStore(310): Store=ee46c734c4c3f049b054ec6f34f49d5c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:55:04,898 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:04,899 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:04,900 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-07 08:55:04,901 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:04,902 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 08:55:04,903 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ee46c734c4c3f049b054ec6f34f49d5c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73884625, jitterRate=0.10096670687198639}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-07 08:55:04,903 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ee46c734c4c3f049b054ec6f34f49d5c: 2023-08-07 08:55:04,903 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c., pid=143, masterSystemTime=1691398504889 2023-08-07 08:55:04,904 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c. 2023-08-07 08:55:04,905 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c. 2023-08-07 08:55:04,905 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=ee46c734c4c3f049b054ec6f34f49d5c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:04,905 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1691398504905"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691398504905"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691398504905"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398504905"}]},"ts":"1691398504905"} 2023-08-07 08:55:04,907 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=143, resume processing ppid=142 2023-08-07 08:55:04,907 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure ee46c734c4c3f049b054ec6f34f49d5c, server=jenkins-hbase4.apache.org,34559,1691398396467 in 168 msec 2023-08-07 08:55:04,909 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=142, resume processing ppid=141 2023-08-07 08:55:04,909 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=ee46c734c4c3f049b054ec6f34f49d5c, ASSIGN in 324 msec 2023-08-07 08:55:04,909 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-07 08:55:04,909 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398504909"}]},"ts":"1691398504909"} 2023-08-07 08:55:04,910 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-08-07 08:55:04,917 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-08-07 08:55:04,918 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 368 msec 2023-08-07 08:55:05,056 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:55:05,057 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-08-07 08:55:05,057 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-08-07 08:55:05,057 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:05,058 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-08-07 08:55:05,058 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-08-07 08:55:05,058 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,058 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,059 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:05,059 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:05,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:05,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:05,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:05,059 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:05,059 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:05,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:05,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:05,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,060 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,060 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:55:05,060 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,060 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,060 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:55:05,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,060 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-07 08:55:05,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:55:05,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:55:05,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,061 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-07 08:55:05,060 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-07 08:55:05,061 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-07 08:55:05,061 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:05,061 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-07 08:55:05,061 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-07 08:55:05,061 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-07 08:55:05,061 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-07 08:55:05,061 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-07 08:55:05,061 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-07 08:55:05,061 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:05,061 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-07 08:55:05,061 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-07 08:55:05,061 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-07 08:55:05,061 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:05,061 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-07 08:55:05,061 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-07 08:55:05,061 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-07 08:55:05,061 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:55:05,061 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c. 2023-08-07 08:55:05,061 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:55:05,061 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c. started... 2023-08-07 08:55:05,062 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ee46c734c4c3f049b054ec6f34f49d5c 2/2 column families, dataSize=120 B heapSize=928 B 2023-08-07 08:55:05,062 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,063 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,063 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,063 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:55:05,064 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,064 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:55:05,071 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/.tmp/cf1/9b60b41babdb445cb831ba1afe9cc3c5 2023-08-07 08:55:05,081 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/.tmp/cf2/fdc509227c7d46c8be6d0bcc598e74fe 2023-08-07 08:55:05,085 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/.tmp/cf1/9b60b41babdb445cb831ba1afe9cc3c5 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/9b60b41babdb445cb831ba1afe9cc3c5 2023-08-07 08:55:05,088 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/9b60b41babdb445cb831ba1afe9cc3c5, entries=2, sequenceid=6, filesize=4.8 K 2023-08-07 08:55:05,089 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/.tmp/cf2/fdc509227c7d46c8be6d0bcc598e74fe as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/fdc509227c7d46c8be6d0bcc598e74fe 2023-08-07 08:55:05,091 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/fdc509227c7d46c8be6d0bcc598e74fe, entries=2, sequenceid=6, filesize=4.8 K 2023-08-07 08:55:05,092 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for ee46c734c4c3f049b054ec6f34f49d5c in 30ms, sequenceid=6, compaction requested=false 2023-08-07 08:55:05,092 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-08-07 08:55:05,092 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ee46c734c4c3f049b054ec6f34f49d5c: 2023-08-07 08:55:05,093 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c. 2023-08-07 08:55:05,093 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:55:05,093 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:55:05,093 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,093 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-07 08:55:05,093 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-07 08:55:05,095 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,095 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:55:05,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:55:05,095 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,095 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:55:05,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:55:05,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:55:05,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-07 08:55:05,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:55:05,097 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-08-07 08:55:05,097 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5e3d0df[Count = 0] remaining members to acquire global barrier 2023-08-07 08:55:05,097 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-08-07 08:55:05,097 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,099 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,099 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,099 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,099 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:55:05,099 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,099 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-07 08:55:05,099 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-07 08:55:05,099 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-07 08:55:05,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,099 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:55:05,099 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-07 08:55:05,099 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-07 08:55:05,099 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:55:05,099 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-07 08:55:05,099 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:55:05,101 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-07 08:55:05,101 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,101 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-07 08:55:05,101 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:05,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:55:05,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:55:05,101 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:05,101 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-07 08:55:05,101 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-07 08:55:05,105 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-07 08:55:05,105 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:05,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:55:05,105 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-07 08:55:05,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:55:05,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-07 08:55:05,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:55:05,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-07 08:55:05,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,34559,1691398396467': 2023-08-07 08:55:05,108 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-08-07 08:55:05,108 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-08-07 08:55:05,108 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:55:05,108 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:55:05,108 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-08-07 08:55:05,108 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:55:05,109 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,109 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,109 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:05,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,109 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,109 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:55:05,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:55:05,109 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:05,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:05,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:05,109 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:05,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:55:05,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,110 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-07 08:55:05,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:05,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:05,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:05,110 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:05,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:55:05,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,111 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-07 08:55:05,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:55:05,112 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-07 08:55:05,112 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,113 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,115 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:05,115 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:05,115 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:05,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:05,115 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:05,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:05,115 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:05,115 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:05,115 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-08-07 08:55:05,116 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:55:05,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:05,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:05,115 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:05,115 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:05,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:05,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:05,116 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:05,116 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:05,116 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-08-07 08:55:05,116 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:55:05,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:05,116 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:05,117 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:05,117 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,117 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,117 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,117 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,117 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,117 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,216 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:55:05,217 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:55:05,227 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:55:05,228 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-08-07 08:55:05,228 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-08-07 08:55:05,228 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-08-07 08:55:05,228 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:05,228 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-08-07 08:55:05,228 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-08-07 08:55:05,228 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,229 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,230 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:05,230 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:05,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:05,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:05,230 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:05,230 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:05,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:05,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:05,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:05,230 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,230 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:55:05,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,231 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,231 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,231 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:55:05,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:55:05,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:55:05,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,231 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-07 08:55:05,231 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-07 08:55:05,231 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-07 08:55:05,232 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-07 08:55:05,232 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-07 08:55:05,232 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-07 08:55:05,232 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-07 08:55:05,232 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-07 08:55:05,232 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-07 08:55:05,232 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-07 08:55:05,232 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:05,232 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:05,232 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-07 08:55:05,232 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-07 08:55:05,232 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-07 08:55:05,232 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-07 08:55:05,232 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-07 08:55:05,232 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-07 08:55:05,232 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-07 08:55:05,232 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-07 08:55:05,232 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:05,232 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-07 08:55:05,232 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:55:05,232 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:55:05,233 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c. 2023-08-07 08:55:05,233 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c. started... 2023-08-07 08:55:05,233 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ee46c734c4c3f049b054ec6f34f49d5c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:55:05,235 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,236 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,236 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,236 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:55:05,237 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,237 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:55:05,241 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/.tmp/cf1/cec48633c0224b26b022b9a3c532a69e 2023-08-07 08:55:05,254 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/.tmp/cf2/9468a53cdc51477a8f99ee801fbfbfbe 2023-08-07 08:55:05,258 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/.tmp/cf1/cec48633c0224b26b022b9a3c532a69e as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/cec48633c0224b26b022b9a3c532a69e 2023-08-07 08:55:05,262 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/cec48633c0224b26b022b9a3c532a69e, entries=10, sequenceid=19, filesize=5.0 K 2023-08-07 08:55:05,262 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/.tmp/cf2/9468a53cdc51477a8f99ee801fbfbfbe as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/9468a53cdc51477a8f99ee801fbfbfbe 2023-08-07 08:55:05,266 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/9468a53cdc51477a8f99ee801fbfbfbe, entries=10, sequenceid=19, filesize=5.0 K 2023-08-07 08:55:05,266 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ee46c734c4c3f049b054ec6f34f49d5c in 33ms, sequenceid=19, compaction requested=false 2023-08-07 08:55:05,266 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ee46c734c4c3f049b054ec6f34f49d5c: 2023-08-07 08:55:05,266 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c. 2023-08-07 08:55:05,266 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:55:05,267 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:55:05,267 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,267 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-07 08:55:05,267 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-07 08:55:05,268 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,268 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:55:05,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:55:05,268 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,268 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:55:05,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:55:05,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:55:05,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-07 08:55:05,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:55:05,270 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-08-07 08:55:05,270 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@503ac5db[Count = 0] remaining members to acquire global barrier 2023-08-07 08:55:05,270 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-08-07 08:55:05,270 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,271 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,271 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,271 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,271 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-07 08:55:05,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,271 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-07 08:55:05,271 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-07 08:55:05,271 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:55:05,271 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-07 08:55:05,271 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:55:05,271 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,272 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:55:05,272 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-07 08:55:05,272 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-07 08:55:05,272 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:55:05,273 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-07 08:55:05,273 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:05,273 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-07 08:55:05,274 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-07 08:55:05,274 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,274 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:05,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,274 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-07 08:55:05,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:55:05,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:55:05,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:55:05,275 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-07 08:55:05,275 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:05,275 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-07 08:55:05,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:55:05,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-07 08:55:05,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:55:05,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-07 08:55:05,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,34559,1691398396467': 2023-08-07 08:55:05,277 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-08-07 08:55:05,277 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-08-07 08:55:05,277 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:55:05,277 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:55:05,277 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-08-07 08:55:05,277 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:55:05,279 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,279 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:55:05,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:55:05,279 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,279 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:05,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,279 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,279 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:05,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,279 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:05,279 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:05,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:05,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:55:05,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:05,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:05,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:05,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:05,280 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-07 08:55:05,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,280 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:55:05,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-07 08:55:05,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,282 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:55:05,282 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-07 08:55:05,282 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,286 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,286 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:05,286 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:05,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:05,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:05,286 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,286 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:05,286 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:05,286 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:05,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:05,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:05,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:05,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:05,286 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:05,286 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,286 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-08-07 08:55:05,286 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:55:05,286 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:05,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:05,286 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:05,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:05,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:05,287 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-08-07 08:55:05,286 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:05,287 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,287 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:55:05,287 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:05,287 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,287 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,287 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,287 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,387 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:55:05,388 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:55:05,398 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:55:05,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-08-07 08:55:05,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-08-07 08:55:05,399 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-08-07 08:55:05,399 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:05,399 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-08-07 08:55:05,399 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-08-07 08:55:05,400 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,400 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,401 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:05,401 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:05,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:05,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:05,401 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,401 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:05,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:05,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:05,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:05,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:05,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,402 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,402 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:55:05,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,402 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,402 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:55:05,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,402 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-07 08:55:05,402 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:55:05,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,403 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-07 08:55:05,403 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-07 08:55:05,403 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-07 08:55:05,403 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-07 08:55:05,403 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-07 08:55:05,403 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:05,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:55:05,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,403 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-07 08:55:05,403 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-07 08:55:05,403 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-07 08:55:05,403 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-07 08:55:05,403 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-07 08:55:05,403 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:05,403 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-07 08:55:05,403 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-07 08:55:05,403 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-07 08:55:05,403 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:55:05,403 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:55:05,403 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c. 2023-08-07 08:55:05,403 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-07 08:55:05,403 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c. started... 2023-08-07 08:55:05,404 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-07 08:55:05,404 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:05,404 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-07 08:55:05,404 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ee46c734c4c3f049b054ec6f34f49d5c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:55:05,404 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-07 08:55:05,404 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-07 08:55:05,404 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,405 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,405 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:55:05,409 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,409 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,409 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:55:05,417 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/.tmp/cf1/273607c92aa046d892ba71005f5721fa 2023-08-07 08:55:05,428 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/.tmp/cf2/a6d5906241f44f428b41b048d288a810 2023-08-07 08:55:05,432 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/.tmp/cf1/273607c92aa046d892ba71005f5721fa as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/273607c92aa046d892ba71005f5721fa 2023-08-07 08:55:05,435 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/273607c92aa046d892ba71005f5721fa, entries=10, sequenceid=32, filesize=5.0 K 2023-08-07 08:55:05,436 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/.tmp/cf2/a6d5906241f44f428b41b048d288a810 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/a6d5906241f44f428b41b048d288a810 2023-08-07 08:55:05,438 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/a6d5906241f44f428b41b048d288a810, entries=10, sequenceid=32, filesize=5.0 K 2023-08-07 08:55:05,439 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ee46c734c4c3f049b054ec6f34f49d5c in 35ms, sequenceid=32, compaction requested=true 2023-08-07 08:55:05,440 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ee46c734c4c3f049b054ec6f34f49d5c: 2023-08-07 08:55:05,440 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c. 2023-08-07 08:55:05,440 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:55:05,440 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:55:05,440 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,440 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-07 08:55:05,440 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-07 08:55:05,442 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,442 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:55:05,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:55:05,442 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,442 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:55:05,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:55:05,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:55:05,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-07 08:55:05,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:55:05,444 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-08-07 08:55:05,444 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7f60e479[Count = 0] remaining members to acquire global barrier 2023-08-07 08:55:05,444 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-08-07 08:55:05,444 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,445 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,445 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,445 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-07 08:55:05,445 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-07 08:55:05,445 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-07 08:55:05,445 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,445 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:55:05,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,445 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,445 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:55:05,445 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-07 08:55:05,445 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:55:05,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,445 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-07 08:55:05,445 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-07 08:55:05,445 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:55:05,446 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-07 08:55:05,446 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:05,446 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-07 08:55:05,448 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-07 08:55:05,448 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-07 08:55:05,448 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,448 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:05,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,448 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-07 08:55:05,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:55:05,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:55:05,448 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:05,448 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-07 08:55:05,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:55:05,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:55:05,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-07 08:55:05,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:55:05,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-07 08:55:05,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,34559,1691398396467': 2023-08-07 08:55:05,451 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-08-07 08:55:05,451 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-08-07 08:55:05,451 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:55:05,451 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:55:05,451 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-08-07 08:55:05,451 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:55:05,452 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,452 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,453 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,452 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,453 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:05,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:55:05,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:55:05,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,453 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:05,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:05,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:05,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:05,453 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:05,453 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:05,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:55:05,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:05,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:05,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,454 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-07 08:55:05,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,454 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:55:05,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-07 08:55:05,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:55:05,455 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-07 08:55:05,456 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,456 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,458 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,458 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,458 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,458 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:05,458 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:05,459 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,458 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:05,459 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,459 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:05,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:05,459 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:05,459 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:05,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:05,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:05,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:05,459 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,459 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-08-07 08:55:05,459 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:55:05,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:05,459 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:05,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:05,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:05,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:05,459 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:05,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:05,459 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-08-07 08:55:05,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:05,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:05,459 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,460 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:55:05,460 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,460 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,460 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,560 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:55:05,561 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:55:05,571 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:55:05,572 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-08-07 08:55:05,572 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-08-07 08:55:05,572 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-08-07 08:55:05,572 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:05,572 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-08-07 08:55:05,572 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-08-07 08:55:05,573 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,573 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,574 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:05,574 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:05,574 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,574 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:05,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:05,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:05,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:05,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:05,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:05,574 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,574 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:55:05,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:05,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,575 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,575 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,575 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:55:05,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:55:05,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,575 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-07 08:55:05,575 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-07 08:55:05,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:55:05,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,575 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-07 08:55:05,576 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-07 08:55:05,576 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-07 08:55:05,576 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-07 08:55:05,576 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-07 08:55:05,576 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-07 08:55:05,576 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-07 08:55:05,576 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:05,576 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-07 08:55:05,576 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-07 08:55:05,576 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:05,576 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-07 08:55:05,576 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-07 08:55:05,576 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-07 08:55:05,576 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:05,576 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-07 08:55:05,576 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-07 08:55:05,576 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:55:05,576 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:55:05,576 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c. 2023-08-07 08:55:05,576 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-07 08:55:05,576 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-07 08:55:05,576 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-07 08:55:05,576 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c. started... 2023-08-07 08:55:05,576 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ee46c734c4c3f049b054ec6f34f49d5c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:55:05,578 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,579 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,579 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,579 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:55:05,579 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,579 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:55:05,585 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/.tmp/cf1/c22b7ce84f684210984161264706990d 2023-08-07 08:55:05,595 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/.tmp/cf2/dcc4b5972743447992751542a226bcf9 2023-08-07 08:55:05,598 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/.tmp/cf1/c22b7ce84f684210984161264706990d as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/c22b7ce84f684210984161264706990d 2023-08-07 08:55:05,601 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/c22b7ce84f684210984161264706990d, entries=10, sequenceid=45, filesize=5.0 K 2023-08-07 08:55:05,602 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/.tmp/cf2/dcc4b5972743447992751542a226bcf9 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/dcc4b5972743447992751542a226bcf9 2023-08-07 08:55:05,604 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/dcc4b5972743447992751542a226bcf9, entries=10, sequenceid=45, filesize=5.0 K 2023-08-07 08:55:05,605 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ee46c734c4c3f049b054ec6f34f49d5c in 29ms, sequenceid=45, compaction requested=true 2023-08-07 08:55:05,605 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ee46c734c4c3f049b054ec6f34f49d5c: 2023-08-07 08:55:05,605 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c. 2023-08-07 08:55:05,605 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:55:05,605 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:55:05,605 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,605 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-07 08:55:05,605 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-07 08:55:05,607 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,607 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:55:05,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:55:05,607 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,607 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:55:05,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:55:05,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:55:05,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-07 08:55:05,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:55:05,609 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-08-07 08:55:05,609 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2154a0e9[Count = 0] remaining members to acquire global barrier 2023-08-07 08:55:05,609 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-08-07 08:55:05,609 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,610 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,610 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,610 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,610 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-07 08:55:05,610 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-07 08:55:05,610 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-07 08:55:05,610 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,611 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:55:05,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,611 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:55:05,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,610 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-07 08:55:05,611 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-07 08:55:05,611 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-07 08:55:05,611 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:55:05,611 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:55:05,613 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,613 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-07 08:55:05,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:55:05,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:55:05,613 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-07 08:55:05,613 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:05,613 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:05,613 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-07 08:55:05,613 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:05,613 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-07 08:55:05,613 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-07 08:55:05,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:55:05,613 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-07 08:55:05,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:55:05,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-07 08:55:05,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:55:05,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-07 08:55:05,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,34559,1691398396467': 2023-08-07 08:55:05,617 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-08-07 08:55:05,617 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-08-07 08:55:05,617 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:55:05,617 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:55:05,617 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-08-07 08:55:05,617 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:55:05,619 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,619 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:55:05,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:55:05,619 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,619 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,619 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:05,619 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:05,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,619 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:05,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:55:05,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:05,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:05,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-07 08:55:05,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:05,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:05,620 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:55:05,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:05,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:05,620 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-07 08:55:05,621 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:55:05,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-07 08:55:05,623 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,623 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,623 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,629 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:05,629 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,629 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:05,629 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:05,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:05,629 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:05,629 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:05,629 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-08-07 08:55:05,629 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:55:05,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:05,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:05,629 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:05,629 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:05,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:05,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:05,629 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:05,630 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-08-07 08:55:05,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:05,629 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:05,629 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:05,630 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:05,630 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:55:05,630 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,630 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,630 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,630 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,630 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,630 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:05,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:05,730 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:55:05,731 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:55:05,741 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:55:05,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-08-07 08:55:05,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-08-07 08:55:05,742 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-08-07 08:55:05,743 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:05,743 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-08-07 08:55:05,743 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-08-07 08:55:05,743 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,743 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,745 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:05,745 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:05,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:05,745 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:05,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:05,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:05,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:05,745 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:05,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:05,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,745 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,745 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:55:05,745 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,746 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,746 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:55:05,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:55:05,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:55:05,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:05,746 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-07 08:55:05,746 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-07 08:55:05,746 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-07 08:55:05,746 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-07 08:55:05,746 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-07 08:55:05,746 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-07 08:55:05,746 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-07 08:55:05,746 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:05,746 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-07 08:55:05,746 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-07 08:55:05,747 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-07 08:55:05,747 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-07 08:55:05,746 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-07 08:55:05,747 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-07 08:55:05,747 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-07 08:55:05,747 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:05,747 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-07 08:55:05,747 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:05,747 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-07 08:55:05,747 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-07 08:55:05,747 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-07 08:55:05,747 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-07 08:55:05,747 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:55:05,747 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:55:05,747 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c. 2023-08-07 08:55:05,747 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c. started... 2023-08-07 08:55:05,747 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ee46c734c4c3f049b054ec6f34f49d5c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:55:05,748 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,748 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,748 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,749 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:55:05,749 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,749 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:55:05,946 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/.tmp/cf1/c68a9d6287694d9f8cb3baaa66d5c606 2023-08-07 08:55:05,965 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/.tmp/cf2/06347138db5e40bfb279b6aca6133168 2023-08-07 08:55:05,968 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/.tmp/cf1/c68a9d6287694d9f8cb3baaa66d5c606 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/c68a9d6287694d9f8cb3baaa66d5c606 2023-08-07 08:55:05,971 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/c68a9d6287694d9f8cb3baaa66d5c606, entries=10, sequenceid=58, filesize=5.0 K 2023-08-07 08:55:05,972 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/.tmp/cf2/06347138db5e40bfb279b6aca6133168 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/06347138db5e40bfb279b6aca6133168 2023-08-07 08:55:05,975 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/06347138db5e40bfb279b6aca6133168, entries=10, sequenceid=58, filesize=5.0 K 2023-08-07 08:55:05,977 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ee46c734c4c3f049b054ec6f34f49d5c in 230ms, sequenceid=58, compaction requested=true 2023-08-07 08:55:05,977 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ee46c734c4c3f049b054ec6f34f49d5c: 2023-08-07 08:55:05,977 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c. 2023-08-07 08:55:05,977 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:55:05,977 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:55:05,977 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,977 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-07 08:55:05,977 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-07 08:55:05,979 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,979 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:55:05,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:55:05,979 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,979 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:55:05,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:55:05,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:55:05,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-07 08:55:05,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:55:05,981 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-08-07 08:55:05,981 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3aaaf934[Count = 0] remaining members to acquire global barrier 2023-08-07 08:55:05,981 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-08-07 08:55:05,981 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,983 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,983 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,983 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,983 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-07 08:55:05,983 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,983 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:55:05,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:05,984 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-07 08:55:05,984 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-07 08:55:05,983 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-07 08:55:05,984 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:55:05,984 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-07 08:55:05,984 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-07 08:55:05,984 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:55:05,984 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:55:05,985 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-07 08:55:05,985 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:05,985 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-07 08:55:05,986 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-07 08:55:05,986 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,986 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:05,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:55:05,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:55:05,986 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-07 08:55:05,986 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:05,986 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-07 08:55:05,986 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-07 08:55:05,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:55:05,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:55:05,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-07 08:55:05,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:55:05,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-07 08:55:05,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,34559,1691398396467': 2023-08-07 08:55:05,990 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-08-07 08:55:05,990 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-08-07 08:55:05,990 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:55:05,990 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:55:05,990 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-08-07 08:55:05,990 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:55:05,991 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,991 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,991 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,991 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:55:05,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:55:05,991 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:05,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,991 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:05,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,991 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:05,992 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:55:05,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:05,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:05,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:05,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:05,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:05,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:05,992 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-07 08:55:05,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:05,993 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:55:05,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-07 08:55:05,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,996 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,996 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:55:05,997 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:05,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-07 08:55:05,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:05,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:05,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:06,000 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:06,000 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:06,000 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:06,000 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:06,000 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:06,000 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:06,000 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:06,000 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:06,000 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:06,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:06,000 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:06,000 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-08-07 08:55:06,001 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:55:06,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:06,000 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:06,001 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:06,001 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:06,001 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:06,001 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:06,001 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:06,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:06,000 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:06,000 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:06,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:06,001 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-08-07 08:55:06,001 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:55:06,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:06,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:06,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:06,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:06,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:06,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:06,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:06,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:06,101 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:55:06,103 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:55:06,114 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-07 08:55:06,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-08-07 08:55:06,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-08-07 08:55:06,116 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-08-07 08:55:06,116 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:06,116 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-08-07 08:55:06,116 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-08-07 08:55:06,116 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:06,117 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:06,118 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:06,118 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:06,118 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:06,118 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:06,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:06,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:06,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:06,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:06,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:06,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:06,118 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:06,118 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-07 08:55:06,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:06,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:06,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:06,119 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:06,119 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:06,119 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:06,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:55:06,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:06,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:55:06,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:06,120 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-07 08:55:06,120 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-07 08:55:06,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-07 08:55:06,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:06,120 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-07 08:55:06,120 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-07 08:55:06,120 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-07 08:55:06,120 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-07 08:55:06,120 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-07 08:55:06,120 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-07 08:55:06,120 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-07 08:55:06,120 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-07 08:55:06,120 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-07 08:55:06,120 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:06,120 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-07 08:55:06,120 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:06,120 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-07 08:55:06,120 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-07 08:55:06,120 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-07 08:55:06,120 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-07 08:55:06,121 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34633,1691398396427' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-07 08:55:06,120 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-07 08:55:06,121 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-07 08:55:06,121 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46525,1691398396377' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-07 08:55:06,121 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-07 08:55:06,121 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-07 08:55:06,121 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c. 2023-08-07 08:55:06,121 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c. started... 2023-08-07 08:55:06,121 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ee46c734c4c3f049b054ec6f34f49d5c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-07 08:55:06,122 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:06,122 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:06,122 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:06,122 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:55:06,123 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:06,123 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:55:06,530 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/.tmp/cf1/a72ba5bad0264a5183fbf79c7ae30801 2023-08-07 08:55:06,541 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/.tmp/cf2/10b9da34b79345f28c8f6ae4106e9861 2023-08-07 08:55:06,545 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/.tmp/cf1/a72ba5bad0264a5183fbf79c7ae30801 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/a72ba5bad0264a5183fbf79c7ae30801 2023-08-07 08:55:06,548 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/a72ba5bad0264a5183fbf79c7ae30801, entries=10, sequenceid=71, filesize=5.0 K 2023-08-07 08:55:06,549 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/.tmp/cf2/10b9da34b79345f28c8f6ae4106e9861 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/10b9da34b79345f28c8f6ae4106e9861 2023-08-07 08:55:06,552 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/10b9da34b79345f28c8f6ae4106e9861, entries=10, sequenceid=71, filesize=5.0 K 2023-08-07 08:55:06,552 INFO [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ee46c734c4c3f049b054ec6f34f49d5c in 431ms, sequenceid=71, compaction requested=true 2023-08-07 08:55:06,553 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ee46c734c4c3f049b054ec6f34f49d5c: 2023-08-07 08:55:06,553 DEBUG [rs(jenkins-hbase4.apache.org,34559,1691398396467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c. 2023-08-07 08:55:06,553 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-07 08:55:06,553 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-07 08:55:06,553 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:06,553 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-07 08:55:06,553 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-07 08:55:06,556 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:06,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:06,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:55:06,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:55:06,556 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:06,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:55:06,556 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:06,556 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-07 08:55:06,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:55:06,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-07 08:55:06,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:06,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:06,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:06,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:55:06,558 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34559,1691398396467' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-08-07 08:55:06,558 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3c38f7a7[Count = 0] remaining members to acquire global barrier 2023-08-07 08:55:06,558 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-08-07 08:55:06,558 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:06,559 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:06,559 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:06,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:06,559 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:06,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:06,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:06,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:06,559 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:06,559 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-07 08:55:06,559 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-07 08:55:06,559 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34633,1691398396427' in zk 2023-08-07 08:55:06,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:06,559 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-07 08:55:06,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:06,559 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-07 08:55:06,559 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-07 08:55:06,559 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-07 08:55:06,559 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,46525,1691398396377' in zk 2023-08-07 08:55:06,559 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-07 08:55:06,559 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34559,1691398396467' in zk 2023-08-07 08:55:06,562 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-07 08:55:06,563 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:06,563 DEBUG [member: 'jenkins-hbase4.apache.org,34633,1691398396427' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-07 08:55:06,570 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-07 08:55:06,570 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:06,570 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:06,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:06,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:55:06,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:55:06,570 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-07 08:55:06,570 DEBUG [member: 'jenkins-hbase4.apache.org,34559,1691398396467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-07 08:55:06,570 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:06,570 DEBUG [member: 'jenkins-hbase4.apache.org,46525,1691398396377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-07 08:55:06,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:55:06,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:55:06,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-07 08:55:06,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:06,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:06,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:06,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:55:06,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-07 08:55:06,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:06,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:06,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:06,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,34559,1691398396467': 2023-08-07 08:55:06,573 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34559,1691398396467' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-08-07 08:55:06,573 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-08-07 08:55:06,573 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-07 08:55:06,573 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-07 08:55:06,573 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-08-07 08:55:06,573 INFO [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-07 08:55:06,575 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:06,575 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:06,575 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:06,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:06,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:06,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:06,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:06,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:06,575 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:06,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-07 08:55:06,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-07 08:55:06,575 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:06,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:06,575 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:06,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:06,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:06,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:06,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-07 08:55:06,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:06,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:06,576 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:06,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-07 08:55:06,576 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:06,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-07 08:55:06,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-07 08:55:06,577 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:06,577 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:06,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:06,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:06,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:06,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:06,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:06,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:06,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:06,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:06,578 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:06,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-07 08:55:06,578 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:06,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-07 08:55:06,578 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:06,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:06,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:06,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:06,581 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:06,581 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:06,581 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:06,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:06,581 DEBUG [(jenkins-hbase4.apache.org,33357,1691398395153)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-07 08:55:06,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:06,581 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:06,581 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:06,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:06,581 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-08-07 08:55:06,581 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-07 08:55:06,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:06,581 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:06,581 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:06,582 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:06,582 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-07 08:55:06,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:06,582 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-08-07 08:55:06,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:06,582 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:06,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:06,582 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:06,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:06,582 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-07 08:55:06,582 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:06,582 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:06,582 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-07 08:55:06,582 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-07 08:55:06,582 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-07 08:55:06,582 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-07 08:55:06,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-07 08:55:06,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-07 08:55:06,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-07 08:55:06,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-07 08:55:06,682 DEBUG [Listener at localhost/33561] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-07 08:55:06,683 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-07 08:55:06,690 DEBUG [Listener at localhost/33561] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:06,691 DEBUG [Listener at localhost/33561] procedure2.ProcedureExecutor(1029): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=ee46c734c4c3f049b054ec6f34f49d5c, daughterA=cf538e5876ec330ccb93fc1a6d516e71, daughterB=7e7041dd5d1c44daa984633ad6291f9c 2023-08-07 08:55:06,692 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=ee46c734c4c3f049b054ec6f34f49d5c, daughterA=cf538e5876ec330ccb93fc1a6d516e71, daughterB=7e7041dd5d1c44daa984633ad6291f9c 2023-08-07 08:55:06,692 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=ee46c734c4c3f049b054ec6f34f49d5c, daughterA=cf538e5876ec330ccb93fc1a6d516e71, daughterB=7e7041dd5d1c44daa984633ad6291f9c 2023-08-07 08:55:06,692 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=ee46c734c4c3f049b054ec6f34f49d5c, daughterA=cf538e5876ec330ccb93fc1a6d516e71, daughterB=7e7041dd5d1c44daa984633ad6291f9c 2023-08-07 08:55:06,694 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=ee46c734c4c3f049b054ec6f34f49d5c, UNASSIGN}] 2023-08-07 08:55:06,695 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=ee46c734c4c3f049b054ec6f34f49d5c, UNASSIGN 2023-08-07 08:55:06,695 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=ee46c734c4c3f049b054ec6f34f49d5c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:06,695 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1691398506695"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398506695"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398506695"}]},"ts":"1691398506695"} 2023-08-07 08:55:06,696 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure ee46c734c4c3f049b054ec6f34f49d5c, server=jenkins-hbase4.apache.org,34559,1691398396467}] 2023-08-07 08:55:06,848 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:06,848 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ee46c734c4c3f049b054ec6f34f49d5c, disabling compactions & flushes 2023-08-07 08:55:06,848 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c. 2023-08-07 08:55:06,848 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c. 2023-08-07 08:55:06,848 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c. after waiting 0 ms 2023-08-07 08:55:06,848 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c. 2023-08-07 08:55:06,848 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing ee46c734c4c3f049b054ec6f34f49d5c 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-08-07 08:55:06,861 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/.tmp/cf1/43d72c1260c54bb8ab9ac76699e3a21a 2023-08-07 08:55:06,872 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/.tmp/cf2/8ed5067df3804d1c9e19e225c3c77c3f 2023-08-07 08:55:06,875 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/.tmp/cf1/43d72c1260c54bb8ab9ac76699e3a21a as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/43d72c1260c54bb8ab9ac76699e3a21a 2023-08-07 08:55:06,879 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/43d72c1260c54bb8ab9ac76699e3a21a, entries=8, sequenceid=82, filesize=5.0 K 2023-08-07 08:55:06,879 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/.tmp/cf2/8ed5067df3804d1c9e19e225c3c77c3f as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/8ed5067df3804d1c9e19e225c3c77c3f 2023-08-07 08:55:06,883 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/8ed5067df3804d1c9e19e225c3c77c3f, entries=8, sequenceid=82, filesize=5.0 K 2023-08-07 08:55:06,883 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for ee46c734c4c3f049b054ec6f34f49d5c in 35ms, sequenceid=82, compaction requested=true 2023-08-07 08:55:06,887 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-08-07 08:55:06,888 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-07 08:55:06,888 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c. 2023-08-07 08:55:06,888 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ee46c734c4c3f049b054ec6f34f49d5c: 2023-08-07 08:55:06,889 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:06,890 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=ee46c734c4c3f049b054ec6f34f49d5c, regionState=CLOSED 2023-08-07 08:55:06,890 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1691398506890"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398506890"}]},"ts":"1691398506890"} 2023-08-07 08:55:06,892 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=146, resume processing ppid=145 2023-08-07 08:55:06,892 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure ee46c734c4c3f049b054ec6f34f49d5c, server=jenkins-hbase4.apache.org,34559,1691398396467 in 195 msec 2023-08-07 08:55:06,893 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=145, resume processing ppid=144 2023-08-07 08:55:06,893 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=ee46c734c4c3f049b054ec6f34f49d5c, UNASSIGN in 198 msec 2023-08-07 08:55:06,902 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=ee46c734c4c3f049b054ec6f34f49d5c, threads=14 2023-08-07 08:55:06,902 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/06347138db5e40bfb279b6aca6133168 for region: ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:06,902 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/10b9da34b79345f28c8f6ae4106e9861 for region: ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:06,902 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/8ed5067df3804d1c9e19e225c3c77c3f for region: ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:06,902 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/9468a53cdc51477a8f99ee801fbfbfbe for region: ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:06,902 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/a6d5906241f44f428b41b048d288a810 for region: ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:06,902 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/dcc4b5972743447992751542a226bcf9 for region: ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:06,902 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/fdc509227c7d46c8be6d0bcc598e74fe for region: ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:06,903 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/273607c92aa046d892ba71005f5721fa for region: ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:06,903 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/43d72c1260c54bb8ab9ac76699e3a21a for region: ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:06,903 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/9b60b41babdb445cb831ba1afe9cc3c5 for region: ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:06,906 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/a72ba5bad0264a5183fbf79c7ae30801 for region: ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:06,907 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/c22b7ce84f684210984161264706990d for region: ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:06,907 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/c68a9d6287694d9f8cb3baaa66d5c606 for region: ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:06,907 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/cec48633c0224b26b022b9a3c532a69e for region: ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:07,031 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-08-07 08:55:07,087 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/fdc509227c7d46c8be6d0bcc598e74fe for region: ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:07,088 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/a6d5906241f44f428b41b048d288a810 for region: ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:07,119 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/10b9da34b79345f28c8f6ae4106e9861 for region: ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:07,120 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/a72ba5bad0264a5183fbf79c7ae30801 for region: ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:07,122 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/9b60b41babdb445cb831ba1afe9cc3c5 for region: ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:07,146 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/cec48633c0224b26b022b9a3c532a69e for region: ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:07,157 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/dcc4b5972743447992751542a226bcf9 for region: ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:07,159 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/06347138db5e40bfb279b6aca6133168 for region: ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:07,178 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/43d72c1260c54bb8ab9ac76699e3a21a for region: ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:07,196 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/c22b7ce84f684210984161264706990d for region: ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:07,197 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/273607c92aa046d892ba71005f5721fa for region: ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:07,198 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/8ed5067df3804d1c9e19e225c3c77c3f for region: ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:07,199 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/9468a53cdc51477a8f99ee801fbfbfbe for region: ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:07,200 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/c68a9d6287694d9f8cb3baaa66d5c606 for region: ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:07,200 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region ee46c734c4c3f049b054ec6f34f49d5c Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-08-07 08:55:07,232 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-08-07 08:55:07,234 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-08-07 08:55:07,235 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1691398507235"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1691398507235"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1691398507235"}]},"ts":"1691398507235"} 2023-08-07 08:55:07,235 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691398507235"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398507235"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398507235"}]},"ts":"1691398507235"} 2023-08-07 08:55:07,235 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691398507235"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398507235"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398507235"}]},"ts":"1691398507235"} 2023-08-07 08:55:07,238 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=cf538e5876ec330ccb93fc1a6d516e71, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=7e7041dd5d1c44daa984633ad6291f9c, ASSIGN}] 2023-08-07 08:55:07,239 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=7e7041dd5d1c44daa984633ad6291f9c, ASSIGN 2023-08-07 08:55:07,239 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=7e7041dd5d1c44daa984633ad6291f9c, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,34559,1691398396467; forceNewPlan=false, retain=false 2023-08-07 08:55:07,240 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=cf538e5876ec330ccb93fc1a6d516e71, ASSIGN 2023-08-07 08:55:07,241 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=cf538e5876ec330ccb93fc1a6d516e71, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,34559,1691398396467; forceNewPlan=false, retain=false 2023-08-07 08:55:07,390 INFO [jenkins-hbase4:33357] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-08-07 08:55:07,391 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=cf538e5876ec330ccb93fc1a6d516e71, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:07,392 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691398507391"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398507391"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398507391"}]},"ts":"1691398507391"} 2023-08-07 08:55:07,393 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=149, ppid=147, state=RUNNABLE; OpenRegionProcedure cf538e5876ec330ccb93fc1a6d516e71, server=jenkins-hbase4.apache.org,34559,1691398396467}] 2023-08-07 08:55:07,395 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=7e7041dd5d1c44daa984633ad6291f9c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:07,395 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691398507395"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398507395"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398507395"}]},"ts":"1691398507395"} 2023-08-07 08:55:07,396 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=150, ppid=148, state=RUNNABLE; OpenRegionProcedure 7e7041dd5d1c44daa984633ad6291f9c, server=jenkins-hbase4.apache.org,34559,1691398396467}] 2023-08-07 08:55:07,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71. 2023-08-07 08:55:07,547 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => cf538e5876ec330ccb93fc1a6d516e71, NAME => 'testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71.', STARTKEY => '', ENDKEY => '41'} 2023-08-07 08:55:07,548 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-07 08:55:07,548 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion cf538e5876ec330ccb93fc1a6d516e71 2023-08-07 08:55:07,548 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:55:07,548 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for cf538e5876ec330ccb93fc1a6d516e71 2023-08-07 08:55:07,548 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for cf538e5876ec330ccb93fc1a6d516e71 2023-08-07 08:55:07,549 INFO [StoreOpener-cf538e5876ec330ccb93fc1a6d516e71-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region cf538e5876ec330ccb93fc1a6d516e71 2023-08-07 08:55:07,550 DEBUG [StoreOpener-cf538e5876ec330ccb93fc1a6d516e71-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1 2023-08-07 08:55:07,550 DEBUG [StoreOpener-cf538e5876ec330ccb93fc1a6d516e71-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1 2023-08-07 08:55:07,550 INFO [StoreOpener-cf538e5876ec330ccb93fc1a6d516e71-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cf538e5876ec330ccb93fc1a6d516e71 columnFamilyName cf1 2023-08-07 08:55:07,562 DEBUG [StoreOpener-cf538e5876ec330ccb93fc1a6d516e71-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/273607c92aa046d892ba71005f5721fa.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/273607c92aa046d892ba71005f5721fa-bottom 2023-08-07 08:55:07,566 DEBUG [StoreOpener-cf538e5876ec330ccb93fc1a6d516e71-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/43d72c1260c54bb8ab9ac76699e3a21a.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/43d72c1260c54bb8ab9ac76699e3a21a-bottom 2023-08-07 08:55:07,570 DEBUG [StoreOpener-cf538e5876ec330ccb93fc1a6d516e71-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/9b60b41babdb445cb831ba1afe9cc3c5.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/9b60b41babdb445cb831ba1afe9cc3c5-bottom 2023-08-07 08:55:07,574 DEBUG [StoreOpener-cf538e5876ec330ccb93fc1a6d516e71-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/a72ba5bad0264a5183fbf79c7ae30801.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/a72ba5bad0264a5183fbf79c7ae30801-bottom 2023-08-07 08:55:07,578 DEBUG [StoreOpener-cf538e5876ec330ccb93fc1a6d516e71-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/c22b7ce84f684210984161264706990d.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/c22b7ce84f684210984161264706990d-bottom 2023-08-07 08:55:07,581 DEBUG [StoreOpener-cf538e5876ec330ccb93fc1a6d516e71-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/c68a9d6287694d9f8cb3baaa66d5c606.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/c68a9d6287694d9f8cb3baaa66d5c606-bottom 2023-08-07 08:55:07,585 DEBUG [StoreOpener-cf538e5876ec330ccb93fc1a6d516e71-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/cec48633c0224b26b022b9a3c532a69e.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/cec48633c0224b26b022b9a3c532a69e-bottom 2023-08-07 08:55:07,585 INFO [StoreOpener-cf538e5876ec330ccb93fc1a6d516e71-1] regionserver.HStore(310): Store=cf538e5876ec330ccb93fc1a6d516e71/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:55:07,585 INFO [StoreOpener-cf538e5876ec330ccb93fc1a6d516e71-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region cf538e5876ec330ccb93fc1a6d516e71 2023-08-07 08:55:07,586 DEBUG [StoreOpener-cf538e5876ec330ccb93fc1a6d516e71-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2 2023-08-07 08:55:07,586 DEBUG [StoreOpener-cf538e5876ec330ccb93fc1a6d516e71-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2 2023-08-07 08:55:07,586 INFO [StoreOpener-cf538e5876ec330ccb93fc1a6d516e71-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cf538e5876ec330ccb93fc1a6d516e71 columnFamilyName cf2 2023-08-07 08:55:07,600 DEBUG [StoreOpener-cf538e5876ec330ccb93fc1a6d516e71-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/06347138db5e40bfb279b6aca6133168.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/06347138db5e40bfb279b6aca6133168-bottom 2023-08-07 08:55:07,604 DEBUG [StoreOpener-cf538e5876ec330ccb93fc1a6d516e71-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/10b9da34b79345f28c8f6ae4106e9861.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/10b9da34b79345f28c8f6ae4106e9861-bottom 2023-08-07 08:55:07,607 DEBUG [StoreOpener-cf538e5876ec330ccb93fc1a6d516e71-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/8ed5067df3804d1c9e19e225c3c77c3f.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/8ed5067df3804d1c9e19e225c3c77c3f-bottom 2023-08-07 08:55:07,610 DEBUG [StoreOpener-cf538e5876ec330ccb93fc1a6d516e71-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/9468a53cdc51477a8f99ee801fbfbfbe.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/9468a53cdc51477a8f99ee801fbfbfbe-bottom 2023-08-07 08:55:07,614 DEBUG [StoreOpener-cf538e5876ec330ccb93fc1a6d516e71-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/a6d5906241f44f428b41b048d288a810.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/a6d5906241f44f428b41b048d288a810-bottom 2023-08-07 08:55:07,617 DEBUG [StoreOpener-cf538e5876ec330ccb93fc1a6d516e71-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/dcc4b5972743447992751542a226bcf9.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/dcc4b5972743447992751542a226bcf9-bottom 2023-08-07 08:55:07,621 DEBUG [StoreOpener-cf538e5876ec330ccb93fc1a6d516e71-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/fdc509227c7d46c8be6d0bcc598e74fe.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/fdc509227c7d46c8be6d0bcc598e74fe-bottom 2023-08-07 08:55:07,621 INFO [StoreOpener-cf538e5876ec330ccb93fc1a6d516e71-1] regionserver.HStore(310): Store=cf538e5876ec330ccb93fc1a6d516e71/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:55:07,622 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71 2023-08-07 08:55:07,623 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71 2023-08-07 08:55:07,624 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-07 08:55:07,625 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for cf538e5876ec330ccb93fc1a6d516e71 2023-08-07 08:55:07,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened cf538e5876ec330ccb93fc1a6d516e71; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59708218, jitterRate=-0.11027821898460388}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-07 08:55:07,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for cf538e5876ec330ccb93fc1a6d516e71: 2023-08-07 08:55:07,627 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71., pid=149, masterSystemTime=1691398507544 2023-08-07 08:55:07,627 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:55:07,628 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-07 08:55:07,628 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:55:07,628 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-07 08:55:07,628 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71. 2023-08-07 08:55:07,628 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HStore(1912): cf538e5876ec330ccb93fc1a6d516e71/cf1 is initiating minor compaction (all files) 2023-08-07 08:55:07,628 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71. 2023-08-07 08:55:07,628 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HStore(1912): cf538e5876ec330ccb93fc1a6d516e71/cf2 is initiating minor compaction (all files) 2023-08-07 08:55:07,628 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of cf538e5876ec330ccb93fc1a6d516e71/cf1 in testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71. 2023-08-07 08:55:07,628 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HRegion(2259): Starting compaction of cf538e5876ec330ccb93fc1a6d516e71/cf2 in testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71. 2023-08-07 08:55:07,628 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/9b60b41babdb445cb831ba1afe9cc3c5.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/9b60b41babdb445cb831ba1afe9cc3c5-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/cec48633c0224b26b022b9a3c532a69e.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/cec48633c0224b26b022b9a3c532a69e-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/273607c92aa046d892ba71005f5721fa.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/273607c92aa046d892ba71005f5721fa-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/c22b7ce84f684210984161264706990d.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/c22b7ce84f684210984161264706990d-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/c68a9d6287694d9f8cb3baaa66d5c606.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/c68a9d6287694d9f8cb3baaa66d5c606-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/a72ba5bad0264a5183fbf79c7ae30801.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/a72ba5bad0264a5183fbf79c7ae30801-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/43d72c1260c54bb8ab9ac76699e3a21a.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/43d72c1260c54bb8ab9ac76699e3a21a-bottom] into tmpdir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/.tmp, totalSize=34.9 K 2023-08-07 08:55:07,629 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/fdc509227c7d46c8be6d0bcc598e74fe.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/fdc509227c7d46c8be6d0bcc598e74fe-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/9468a53cdc51477a8f99ee801fbfbfbe.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/9468a53cdc51477a8f99ee801fbfbfbe-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/a6d5906241f44f428b41b048d288a810.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/a6d5906241f44f428b41b048d288a810-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/dcc4b5972743447992751542a226bcf9.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/dcc4b5972743447992751542a226bcf9-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/06347138db5e40bfb279b6aca6133168.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/06347138db5e40bfb279b6aca6133168-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/10b9da34b79345f28c8f6ae4106e9861.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/10b9da34b79345f28c8f6ae4106e9861-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/8ed5067df3804d1c9e19e225c3c77c3f.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/8ed5067df3804d1c9e19e225c3c77c3f-bottom] into tmpdir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/.tmp, totalSize=34.9 K 2023-08-07 08:55:07,629 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting 9b60b41babdb445cb831ba1afe9cc3c5.ee46c734c4c3f049b054ec6f34f49d5c, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1691398505052 2023-08-07 08:55:07,629 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting fdc509227c7d46c8be6d0bcc598e74fe.ee46c734c4c3f049b054ec6f34f49d5c, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1691398505052 2023-08-07 08:55:07,629 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71. 2023-08-07 08:55:07,629 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71. 2023-08-07 08:55:07,629 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c. 2023-08-07 08:55:07,630 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 7e7041dd5d1c44daa984633ad6291f9c, NAME => 'testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c.', STARTKEY => '41', ENDKEY => ''} 2023-08-07 08:55:07,630 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-07 08:55:07,630 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 7e7041dd5d1c44daa984633ad6291f9c 2023-08-07 08:55:07,630 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 08:55:07,630 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 7e7041dd5d1c44daa984633ad6291f9c 2023-08-07 08:55:07,630 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 7e7041dd5d1c44daa984633ad6291f9c 2023-08-07 08:55:07,631 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=cf538e5876ec330ccb93fc1a6d516e71, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:07,631 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting cec48633c0224b26b022b9a3c532a69e.ee46c734c4c3f049b054ec6f34f49d5c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1691398505218 2023-08-07 08:55:07,631 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691398507631"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691398507631"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691398507631"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398507631"}]},"ts":"1691398507631"} 2023-08-07 08:55:07,631 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting 9468a53cdc51477a8f99ee801fbfbfbe.ee46c734c4c3f049b054ec6f34f49d5c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1691398505218 2023-08-07 08:55:07,631 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting 273607c92aa046d892ba71005f5721fa.ee46c734c4c3f049b054ec6f34f49d5c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1691398505388 2023-08-07 08:55:07,632 INFO [StoreOpener-7e7041dd5d1c44daa984633ad6291f9c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7e7041dd5d1c44daa984633ad6291f9c 2023-08-07 08:55:07,632 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting a6d5906241f44f428b41b048d288a810.ee46c734c4c3f049b054ec6f34f49d5c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1691398505388 2023-08-07 08:55:07,632 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting c22b7ce84f684210984161264706990d.ee46c734c4c3f049b054ec6f34f49d5c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1691398505562 2023-08-07 08:55:07,632 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting c68a9d6287694d9f8cb3baaa66d5c606.ee46c734c4c3f049b054ec6f34f49d5c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1691398505732 2023-08-07 08:55:07,633 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting dcc4b5972743447992751542a226bcf9.ee46c734c4c3f049b054ec6f34f49d5c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1691398505562 2023-08-07 08:55:07,633 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting a72ba5bad0264a5183fbf79c7ae30801.ee46c734c4c3f049b054ec6f34f49d5c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1691398506104 2023-08-07 08:55:07,633 DEBUG [StoreOpener-7e7041dd5d1c44daa984633ad6291f9c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1 2023-08-07 08:55:07,633 DEBUG [StoreOpener-7e7041dd5d1c44daa984633ad6291f9c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1 2023-08-07 08:55:07,633 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting 06347138db5e40bfb279b6aca6133168.ee46c734c4c3f049b054ec6f34f49d5c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1691398505732 2023-08-07 08:55:07,633 INFO [StoreOpener-7e7041dd5d1c44daa984633ad6291f9c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7e7041dd5d1c44daa984633ad6291f9c columnFamilyName cf1 2023-08-07 08:55:07,634 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting 43d72c1260c54bb8ab9ac76699e3a21a.ee46c734c4c3f049b054ec6f34f49d5c, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1691398506684 2023-08-07 08:55:07,634 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting 10b9da34b79345f28c8f6ae4106e9861.ee46c734c4c3f049b054ec6f34f49d5c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1691398506104 2023-08-07 08:55:07,634 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=149, resume processing ppid=147 2023-08-07 08:55:07,634 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=149, ppid=147, state=SUCCESS; OpenRegionProcedure cf538e5876ec330ccb93fc1a6d516e71, server=jenkins-hbase4.apache.org,34559,1691398396467 in 239 msec 2023-08-07 08:55:07,635 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting 8ed5067df3804d1c9e19e225c3c77c3f.ee46c734c4c3f049b054ec6f34f49d5c, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1691398506684 2023-08-07 08:55:07,636 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=cf538e5876ec330ccb93fc1a6d516e71, ASSIGN in 396 msec 2023-08-07 08:55:07,646 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] throttle.PressureAwareThroughputController(145): cf538e5876ec330ccb93fc1a6d516e71#cf1#compaction#140 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-07 08:55:07,648 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] throttle.PressureAwareThroughputController(145): cf538e5876ec330ccb93fc1a6d516e71#cf2#compaction#141 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-07 08:55:07,648 DEBUG [StoreOpener-7e7041dd5d1c44daa984633ad6291f9c-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/273607c92aa046d892ba71005f5721fa.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/273607c92aa046d892ba71005f5721fa-top 2023-08-07 08:55:07,659 DEBUG [StoreOpener-7e7041dd5d1c44daa984633ad6291f9c-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/43d72c1260c54bb8ab9ac76699e3a21a.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/43d72c1260c54bb8ab9ac76699e3a21a-top 2023-08-07 08:55:07,670 DEBUG [StoreOpener-7e7041dd5d1c44daa984633ad6291f9c-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/9b60b41babdb445cb831ba1afe9cc3c5.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/9b60b41babdb445cb831ba1afe9cc3c5-top 2023-08-07 08:55:07,676 DEBUG [StoreOpener-7e7041dd5d1c44daa984633ad6291f9c-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/a72ba5bad0264a5183fbf79c7ae30801.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/a72ba5bad0264a5183fbf79c7ae30801-top 2023-08-07 08:55:07,678 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/.tmp/cf1/00697ed133ff4aeca8cd12bb2966fb51 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/00697ed133ff4aeca8cd12bb2966fb51 2023-08-07 08:55:07,679 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/.tmp/cf2/905852ab1029495fa27db79fa4c532e0 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/905852ab1029495fa27db79fa4c532e0 2023-08-07 08:55:07,680 DEBUG [StoreOpener-7e7041dd5d1c44daa984633ad6291f9c-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/c22b7ce84f684210984161264706990d.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/c22b7ce84f684210984161264706990d-top 2023-08-07 08:55:07,684 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in cf538e5876ec330ccb93fc1a6d516e71/cf1 of cf538e5876ec330ccb93fc1a6d516e71 into 00697ed133ff4aeca8cd12bb2966fb51(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 08:55:07,684 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for cf538e5876ec330ccb93fc1a6d516e71: 2023-08-07 08:55:07,684 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71., storeName=cf538e5876ec330ccb93fc1a6d516e71/cf1, priority=9, startTime=1691398507627; duration=0sec 2023-08-07 08:55:07,684 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:55:07,685 DEBUG [StoreOpener-7e7041dd5d1c44daa984633ad6291f9c-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/c68a9d6287694d9f8cb3baaa66d5c606.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/c68a9d6287694d9f8cb3baaa66d5c606-top 2023-08-07 08:55:07,686 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in cf538e5876ec330ccb93fc1a6d516e71/cf2 of cf538e5876ec330ccb93fc1a6d516e71 into 905852ab1029495fa27db79fa4c532e0(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 08:55:07,686 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for cf538e5876ec330ccb93fc1a6d516e71: 2023-08-07 08:55:07,686 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71., storeName=cf538e5876ec330ccb93fc1a6d516e71/cf2, priority=9, startTime=1691398507628; duration=0sec 2023-08-07 08:55:07,686 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:55:07,689 DEBUG [StoreOpener-7e7041dd5d1c44daa984633ad6291f9c-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/cec48633c0224b26b022b9a3c532a69e.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/cec48633c0224b26b022b9a3c532a69e-top 2023-08-07 08:55:07,689 INFO [StoreOpener-7e7041dd5d1c44daa984633ad6291f9c-1] regionserver.HStore(310): Store=7e7041dd5d1c44daa984633ad6291f9c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:55:07,689 INFO [StoreOpener-7e7041dd5d1c44daa984633ad6291f9c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 7e7041dd5d1c44daa984633ad6291f9c 2023-08-07 08:55:07,690 DEBUG [StoreOpener-7e7041dd5d1c44daa984633ad6291f9c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2 2023-08-07 08:55:07,690 DEBUG [StoreOpener-7e7041dd5d1c44daa984633ad6291f9c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2 2023-08-07 08:55:07,690 INFO [StoreOpener-7e7041dd5d1c44daa984633ad6291f9c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7e7041dd5d1c44daa984633ad6291f9c columnFamilyName cf2 2023-08-07 08:55:07,703 DEBUG [StoreOpener-7e7041dd5d1c44daa984633ad6291f9c-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/06347138db5e40bfb279b6aca6133168.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/06347138db5e40bfb279b6aca6133168-top 2023-08-07 08:55:07,706 DEBUG [StoreOpener-7e7041dd5d1c44daa984633ad6291f9c-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/10b9da34b79345f28c8f6ae4106e9861.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/10b9da34b79345f28c8f6ae4106e9861-top 2023-08-07 08:55:07,709 DEBUG [StoreOpener-7e7041dd5d1c44daa984633ad6291f9c-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/8ed5067df3804d1c9e19e225c3c77c3f.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/8ed5067df3804d1c9e19e225c3c77c3f-top 2023-08-07 08:55:07,712 DEBUG [StoreOpener-7e7041dd5d1c44daa984633ad6291f9c-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/9468a53cdc51477a8f99ee801fbfbfbe.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/9468a53cdc51477a8f99ee801fbfbfbe-top 2023-08-07 08:55:07,715 DEBUG [StoreOpener-7e7041dd5d1c44daa984633ad6291f9c-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/a6d5906241f44f428b41b048d288a810.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/a6d5906241f44f428b41b048d288a810-top 2023-08-07 08:55:07,717 DEBUG [StoreOpener-7e7041dd5d1c44daa984633ad6291f9c-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/dcc4b5972743447992751542a226bcf9.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/dcc4b5972743447992751542a226bcf9-top 2023-08-07 08:55:07,720 DEBUG [StoreOpener-7e7041dd5d1c44daa984633ad6291f9c-1] regionserver.HStore(539): loaded hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/fdc509227c7d46c8be6d0bcc598e74fe.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/fdc509227c7d46c8be6d0bcc598e74fe-top 2023-08-07 08:55:07,720 INFO [StoreOpener-7e7041dd5d1c44daa984633ad6291f9c-1] regionserver.HStore(310): Store=7e7041dd5d1c44daa984633ad6291f9c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 08:55:07,721 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c 2023-08-07 08:55:07,722 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c 2023-08-07 08:55:07,723 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-07 08:55:07,724 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 7e7041dd5d1c44daa984633ad6291f9c 2023-08-07 08:55:07,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 7e7041dd5d1c44daa984633ad6291f9c; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73370018, jitterRate=0.09329846501350403}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-07 08:55:07,724 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 7e7041dd5d1c44daa984633ad6291f9c: 2023-08-07 08:55:07,725 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c., pid=150, masterSystemTime=1691398507544 2023-08-07 08:55:07,725 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:55:07,726 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-07 08:55:07,726 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-07 08:55:07,726 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-07 08:55:07,726 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c. 2023-08-07 08:55:07,726 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HStore(1912): 7e7041dd5d1c44daa984633ad6291f9c/cf1 is initiating minor compaction (all files) 2023-08-07 08:55:07,726 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c. 2023-08-07 08:55:07,726 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 7e7041dd5d1c44daa984633ad6291f9c/cf1 in testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c. 2023-08-07 08:55:07,726 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HStore(1912): 7e7041dd5d1c44daa984633ad6291f9c/cf2 is initiating minor compaction (all files) 2023-08-07 08:55:07,726 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 7e7041dd5d1c44daa984633ad6291f9c/cf2 in testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c. 2023-08-07 08:55:07,727 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/9b60b41babdb445cb831ba1afe9cc3c5.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/9b60b41babdb445cb831ba1afe9cc3c5-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/cec48633c0224b26b022b9a3c532a69e.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/cec48633c0224b26b022b9a3c532a69e-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/273607c92aa046d892ba71005f5721fa.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/273607c92aa046d892ba71005f5721fa-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/c22b7ce84f684210984161264706990d.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/c22b7ce84f684210984161264706990d-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/c68a9d6287694d9f8cb3baaa66d5c606.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/c68a9d6287694d9f8cb3baaa66d5c606-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/a72ba5bad0264a5183fbf79c7ae30801.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/a72ba5bad0264a5183fbf79c7ae30801-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/43d72c1260c54bb8ab9ac76699e3a21a.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/43d72c1260c54bb8ab9ac76699e3a21a-top] into tmpdir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/.tmp, totalSize=34.9 K 2023-08-07 08:55:07,727 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/fdc509227c7d46c8be6d0bcc598e74fe.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/fdc509227c7d46c8be6d0bcc598e74fe-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/9468a53cdc51477a8f99ee801fbfbfbe.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/9468a53cdc51477a8f99ee801fbfbfbe-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/a6d5906241f44f428b41b048d288a810.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/a6d5906241f44f428b41b048d288a810-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/dcc4b5972743447992751542a226bcf9.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/dcc4b5972743447992751542a226bcf9-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/06347138db5e40bfb279b6aca6133168.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/06347138db5e40bfb279b6aca6133168-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/10b9da34b79345f28c8f6ae4106e9861.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/10b9da34b79345f28c8f6ae4106e9861-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/8ed5067df3804d1c9e19e225c3c77c3f.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/8ed5067df3804d1c9e19e225c3c77c3f-top] into tmpdir=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/.tmp, totalSize=34.9 K 2023-08-07 08:55:07,727 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting 9b60b41babdb445cb831ba1afe9cc3c5.ee46c734c4c3f049b054ec6f34f49d5c, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1691398505052 2023-08-07 08:55:07,727 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c. 2023-08-07 08:55:07,727 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting fdc509227c7d46c8be6d0bcc598e74fe.ee46c734c4c3f049b054ec6f34f49d5c, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1691398505052 2023-08-07 08:55:07,727 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c. 2023-08-07 08:55:07,728 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting cec48633c0224b26b022b9a3c532a69e.ee46c734c4c3f049b054ec6f34f49d5c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1691398505218 2023-08-07 08:55:07,728 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=7e7041dd5d1c44daa984633ad6291f9c, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:07,728 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting 9468a53cdc51477a8f99ee801fbfbfbe.ee46c734c4c3f049b054ec6f34f49d5c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1691398505218 2023-08-07 08:55:07,728 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691398507728"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691398507728"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691398507728"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691398507728"}]},"ts":"1691398507728"} 2023-08-07 08:55:07,728 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting 273607c92aa046d892ba71005f5721fa.ee46c734c4c3f049b054ec6f34f49d5c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1691398505388 2023-08-07 08:55:07,728 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting a6d5906241f44f428b41b048d288a810.ee46c734c4c3f049b054ec6f34f49d5c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1691398505388 2023-08-07 08:55:07,728 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting c22b7ce84f684210984161264706990d.ee46c734c4c3f049b054ec6f34f49d5c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1691398505562 2023-08-07 08:55:07,729 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting dcc4b5972743447992751542a226bcf9.ee46c734c4c3f049b054ec6f34f49d5c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1691398505562 2023-08-07 08:55:07,729 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting c68a9d6287694d9f8cb3baaa66d5c606.ee46c734c4c3f049b054ec6f34f49d5c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1691398505732 2023-08-07 08:55:07,729 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting 06347138db5e40bfb279b6aca6133168.ee46c734c4c3f049b054ec6f34f49d5c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1691398505732 2023-08-07 08:55:07,729 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting a72ba5bad0264a5183fbf79c7ae30801.ee46c734c4c3f049b054ec6f34f49d5c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1691398506104 2023-08-07 08:55:07,730 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting 10b9da34b79345f28c8f6ae4106e9861.ee46c734c4c3f049b054ec6f34f49d5c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1691398506104 2023-08-07 08:55:07,730 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] compactions.Compactor(207): Compacting 43d72c1260c54bb8ab9ac76699e3a21a.ee46c734c4c3f049b054ec6f34f49d5c, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1691398506684 2023-08-07 08:55:07,730 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] compactions.Compactor(207): Compacting 8ed5067df3804d1c9e19e225c3c77c3f.ee46c734c4c3f049b054ec6f34f49d5c, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1691398506684 2023-08-07 08:55:07,731 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=150, resume processing ppid=148 2023-08-07 08:55:07,731 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=150, ppid=148, state=SUCCESS; OpenRegionProcedure 7e7041dd5d1c44daa984633ad6291f9c, server=jenkins-hbase4.apache.org,34559,1691398396467 in 333 msec 2023-08-07 08:55:07,733 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=148, resume processing ppid=144 2023-08-07 08:55:07,733 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=7e7041dd5d1c44daa984633ad6291f9c, ASSIGN in 493 msec 2023-08-07 08:55:07,734 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=ee46c734c4c3f049b054ec6f34f49d5c, daughterA=cf538e5876ec330ccb93fc1a6d516e71, daughterB=7e7041dd5d1c44daa984633ad6291f9c in 1.0430 sec 2023-08-07 08:55:07,739 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] throttle.PressureAwareThroughputController(145): 7e7041dd5d1c44daa984633ad6291f9c#cf2#compaction#142 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-08-07 08:55:07,739 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] throttle.PressureAwareThroughputController(145): 7e7041dd5d1c44daa984633ad6291f9c#cf1#compaction#143 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-07 08:55:07,757 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/.tmp/cf1/7d3e573588254b6194cdeea22d25b24a as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/7d3e573588254b6194cdeea22d25b24a 2023-08-07 08:55:07,758 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/.tmp/cf2/1031e67d4c424279bd02e6c8a096e288 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/1031e67d4c424279bd02e6c8a096e288 2023-08-07 08:55:07,761 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 7e7041dd5d1c44daa984633ad6291f9c/cf1 of 7e7041dd5d1c44daa984633ad6291f9c into 7d3e573588254b6194cdeea22d25b24a(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 08:55:07,761 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 7e7041dd5d1c44daa984633ad6291f9c/cf2 of 7e7041dd5d1c44daa984633ad6291f9c into 1031e67d4c424279bd02e6c8a096e288(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-07 08:55:07,762 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 7e7041dd5d1c44daa984633ad6291f9c: 2023-08-07 08:55:07,762 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 7e7041dd5d1c44daa984633ad6291f9c: 2023-08-07 08:55:07,762 INFO [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c., storeName=7e7041dd5d1c44daa984633ad6291f9c/cf1, priority=9, startTime=1691398507725; duration=0sec 2023-08-07 08:55:07,762 INFO [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c., storeName=7e7041dd5d1c44daa984633ad6291f9c/cf2, priority=9, startTime=1691398507726; duration=0sec 2023-08-07 08:55:07,762 DEBUG [RS:2;jenkins-hbase4:34559-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:55:07,762 DEBUG [RS:2;jenkins-hbase4:34559-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-07 08:55:07,942 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34559] ipc.CallRunner(144): callId: 1118 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:45148 deadline: 1691398567941, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c. is not online on jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:08,052 DEBUG [Listener at localhost/33561] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-08-07 08:55:08,059 INFO [Listener at localhost/33561] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-08-07 08:55:08,064 DEBUG [Listener at localhost/33561] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-08-07 08:55:08,070 WARN [Listener at localhost/33561] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-07 08:55:08,071 INFO [Listener at localhost/33561] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-08-07 08:55:08,072 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegion 2023-08-07 08:55:08,072 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure2.ProcedureExecutor(1029): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-08-07 08:55:08,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=151 2023-08-07 08:55:08,075 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398508075"}]},"ts":"1691398508075"} 2023-08-07 08:55:08,076 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-08-07 08:55:08,078 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-08-07 08:55:08,078 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=cf538e5876ec330ccb93fc1a6d516e71, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=7e7041dd5d1c44daa984633ad6291f9c, UNASSIGN}] 2023-08-07 08:55:08,079 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=7e7041dd5d1c44daa984633ad6291f9c, UNASSIGN 2023-08-07 08:55:08,079 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=7e7041dd5d1c44daa984633ad6291f9c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:08,079 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691398508079"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398508079"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398508079"}]},"ts":"1691398508079"} 2023-08-07 08:55:08,080 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure 7e7041dd5d1c44daa984633ad6291f9c, server=jenkins-hbase4.apache.org,34559,1691398396467}] 2023-08-07 08:55:08,082 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=cf538e5876ec330ccb93fc1a6d516e71, UNASSIGN 2023-08-07 08:55:08,082 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=cf538e5876ec330ccb93fc1a6d516e71, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:08,082 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691398508082"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691398508082"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691398508082"}]},"ts":"1691398508082"} 2023-08-07 08:55:08,085 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure cf538e5876ec330ccb93fc1a6d516e71, server=jenkins-hbase4.apache.org,34559,1691398396467}] 2023-08-07 08:55:08,176 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=151 2023-08-07 08:55:08,232 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close cf538e5876ec330ccb93fc1a6d516e71 2023-08-07 08:55:08,232 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing cf538e5876ec330ccb93fc1a6d516e71, disabling compactions & flushes 2023-08-07 08:55:08,232 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71. 2023-08-07 08:55:08,232 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71. 2023-08-07 08:55:08,232 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71. after waiting 0 ms 2023-08-07 08:55:08,232 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71. 2023-08-07 08:55:08,233 DEBUG [StoreCloser-testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/9b60b41babdb445cb831ba1afe9cc3c5.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/9b60b41babdb445cb831ba1afe9cc3c5-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/cec48633c0224b26b022b9a3c532a69e.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/cec48633c0224b26b022b9a3c532a69e-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/273607c92aa046d892ba71005f5721fa.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/273607c92aa046d892ba71005f5721fa-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/c22b7ce84f684210984161264706990d.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/c22b7ce84f684210984161264706990d-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/c68a9d6287694d9f8cb3baaa66d5c606.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/c68a9d6287694d9f8cb3baaa66d5c606-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/a72ba5bad0264a5183fbf79c7ae30801.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/a72ba5bad0264a5183fbf79c7ae30801-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/43d72c1260c54bb8ab9ac76699e3a21a.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/43d72c1260c54bb8ab9ac76699e3a21a-bottom] to archive 2023-08-07 08:55:08,233 DEBUG [StoreCloser-testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-07 08:55:08,235 DEBUG [StoreCloser-testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/9b60b41babdb445cb831ba1afe9cc3c5.ee46c734c4c3f049b054ec6f34f49d5c to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/9b60b41babdb445cb831ba1afe9cc3c5.ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:08,236 DEBUG [StoreCloser-testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/cec48633c0224b26b022b9a3c532a69e.ee46c734c4c3f049b054ec6f34f49d5c to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/cec48633c0224b26b022b9a3c532a69e.ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:08,237 DEBUG [StoreCloser-testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/273607c92aa046d892ba71005f5721fa.ee46c734c4c3f049b054ec6f34f49d5c to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/273607c92aa046d892ba71005f5721fa.ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:08,238 DEBUG [StoreCloser-testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/c22b7ce84f684210984161264706990d.ee46c734c4c3f049b054ec6f34f49d5c to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/c22b7ce84f684210984161264706990d.ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:08,239 DEBUG [StoreCloser-testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/c68a9d6287694d9f8cb3baaa66d5c606.ee46c734c4c3f049b054ec6f34f49d5c to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/c68a9d6287694d9f8cb3baaa66d5c606.ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:08,240 DEBUG [StoreCloser-testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/a72ba5bad0264a5183fbf79c7ae30801.ee46c734c4c3f049b054ec6f34f49d5c to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/a72ba5bad0264a5183fbf79c7ae30801.ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:08,240 DEBUG [StoreCloser-testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/43d72c1260c54bb8ab9ac76699e3a21a.ee46c734c4c3f049b054ec6f34f49d5c to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/43d72c1260c54bb8ab9ac76699e3a21a.ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:08,242 DEBUG [StoreCloser-testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/fdc509227c7d46c8be6d0bcc598e74fe.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/fdc509227c7d46c8be6d0bcc598e74fe-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/9468a53cdc51477a8f99ee801fbfbfbe.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/9468a53cdc51477a8f99ee801fbfbfbe-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/a6d5906241f44f428b41b048d288a810.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/a6d5906241f44f428b41b048d288a810-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/dcc4b5972743447992751542a226bcf9.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/dcc4b5972743447992751542a226bcf9-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/06347138db5e40bfb279b6aca6133168.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/06347138db5e40bfb279b6aca6133168-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/10b9da34b79345f28c8f6ae4106e9861.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/10b9da34b79345f28c8f6ae4106e9861-bottom, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/8ed5067df3804d1c9e19e225c3c77c3f.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/8ed5067df3804d1c9e19e225c3c77c3f-bottom] to archive 2023-08-07 08:55:08,242 DEBUG [StoreCloser-testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-07 08:55:08,244 DEBUG [StoreCloser-testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/fdc509227c7d46c8be6d0bcc598e74fe.ee46c734c4c3f049b054ec6f34f49d5c to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/fdc509227c7d46c8be6d0bcc598e74fe.ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:08,245 DEBUG [StoreCloser-testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/9468a53cdc51477a8f99ee801fbfbfbe.ee46c734c4c3f049b054ec6f34f49d5c to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/9468a53cdc51477a8f99ee801fbfbfbe.ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:08,246 DEBUG [StoreCloser-testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/a6d5906241f44f428b41b048d288a810.ee46c734c4c3f049b054ec6f34f49d5c to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/a6d5906241f44f428b41b048d288a810.ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:08,247 DEBUG [StoreCloser-testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/dcc4b5972743447992751542a226bcf9.ee46c734c4c3f049b054ec6f34f49d5c to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/dcc4b5972743447992751542a226bcf9.ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:08,247 DEBUG [StoreCloser-testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/06347138db5e40bfb279b6aca6133168.ee46c734c4c3f049b054ec6f34f49d5c to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/06347138db5e40bfb279b6aca6133168.ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:08,248 DEBUG [StoreCloser-testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/10b9da34b79345f28c8f6ae4106e9861.ee46c734c4c3f049b054ec6f34f49d5c to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/10b9da34b79345f28c8f6ae4106e9861.ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:08,249 DEBUG [StoreCloser-testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/8ed5067df3804d1c9e19e225c3c77c3f.ee46c734c4c3f049b054ec6f34f49d5c to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/8ed5067df3804d1c9e19e225c3c77c3f.ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:08,252 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-08-07 08:55:08,253 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-07 08:55:08,253 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71. 2023-08-07 08:55:08,253 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for cf538e5876ec330ccb93fc1a6d516e71: 2023-08-07 08:55:08,254 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed cf538e5876ec330ccb93fc1a6d516e71 2023-08-07 08:55:08,254 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 7e7041dd5d1c44daa984633ad6291f9c 2023-08-07 08:55:08,254 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 7e7041dd5d1c44daa984633ad6291f9c, disabling compactions & flushes 2023-08-07 08:55:08,254 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c. 2023-08-07 08:55:08,254 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c. 2023-08-07 08:55:08,254 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=cf538e5876ec330ccb93fc1a6d516e71, regionState=CLOSED 2023-08-07 08:55:08,254 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c. after waiting 0 ms 2023-08-07 08:55:08,254 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c. 2023-08-07 08:55:08,255 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691398508254"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398508254"}]},"ts":"1691398508254"} 2023-08-07 08:55:08,256 DEBUG [StoreCloser-testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/9b60b41babdb445cb831ba1afe9cc3c5.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/9b60b41babdb445cb831ba1afe9cc3c5-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/cec48633c0224b26b022b9a3c532a69e.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/cec48633c0224b26b022b9a3c532a69e-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/273607c92aa046d892ba71005f5721fa.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/273607c92aa046d892ba71005f5721fa-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/c22b7ce84f684210984161264706990d.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/c22b7ce84f684210984161264706990d-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/c68a9d6287694d9f8cb3baaa66d5c606.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/c68a9d6287694d9f8cb3baaa66d5c606-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/a72ba5bad0264a5183fbf79c7ae30801.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/a72ba5bad0264a5183fbf79c7ae30801-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/43d72c1260c54bb8ab9ac76699e3a21a.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/43d72c1260c54bb8ab9ac76699e3a21a-top] to archive 2023-08-07 08:55:08,256 DEBUG [StoreCloser-testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-07 08:55:08,257 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=155, resume processing ppid=152 2023-08-07 08:55:08,257 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure cf538e5876ec330ccb93fc1a6d516e71, server=jenkins-hbase4.apache.org,34559,1691398396467 in 173 msec 2023-08-07 08:55:08,258 DEBUG [StoreCloser-testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/9b60b41babdb445cb831ba1afe9cc3c5.ee46c734c4c3f049b054ec6f34f49d5c to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/9b60b41babdb445cb831ba1afe9cc3c5.ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:08,258 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=cf538e5876ec330ccb93fc1a6d516e71, UNASSIGN in 179 msec 2023-08-07 08:55:08,259 DEBUG [StoreCloser-testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/cec48633c0224b26b022b9a3c532a69e.ee46c734c4c3f049b054ec6f34f49d5c to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/cec48633c0224b26b022b9a3c532a69e.ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:08,260 DEBUG [StoreCloser-testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/273607c92aa046d892ba71005f5721fa.ee46c734c4c3f049b054ec6f34f49d5c to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/273607c92aa046d892ba71005f5721fa.ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:08,261 DEBUG [StoreCloser-testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/c22b7ce84f684210984161264706990d.ee46c734c4c3f049b054ec6f34f49d5c to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/c22b7ce84f684210984161264706990d.ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:08,262 DEBUG [StoreCloser-testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/c68a9d6287694d9f8cb3baaa66d5c606.ee46c734c4c3f049b054ec6f34f49d5c to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/c68a9d6287694d9f8cb3baaa66d5c606.ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:08,263 DEBUG [StoreCloser-testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/a72ba5bad0264a5183fbf79c7ae30801.ee46c734c4c3f049b054ec6f34f49d5c to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/a72ba5bad0264a5183fbf79c7ae30801.ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:08,264 DEBUG [StoreCloser-testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/43d72c1260c54bb8ab9ac76699e3a21a.ee46c734c4c3f049b054ec6f34f49d5c to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/43d72c1260c54bb8ab9ac76699e3a21a.ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:08,266 DEBUG [StoreCloser-testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/fdc509227c7d46c8be6d0bcc598e74fe.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/fdc509227c7d46c8be6d0bcc598e74fe-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/9468a53cdc51477a8f99ee801fbfbfbe.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/9468a53cdc51477a8f99ee801fbfbfbe-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/a6d5906241f44f428b41b048d288a810.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/a6d5906241f44f428b41b048d288a810-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/dcc4b5972743447992751542a226bcf9.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/dcc4b5972743447992751542a226bcf9-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/06347138db5e40bfb279b6aca6133168.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/06347138db5e40bfb279b6aca6133168-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/10b9da34b79345f28c8f6ae4106e9861.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/10b9da34b79345f28c8f6ae4106e9861-top, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/8ed5067df3804d1c9e19e225c3c77c3f.ee46c734c4c3f049b054ec6f34f49d5c->hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/8ed5067df3804d1c9e19e225c3c77c3f-top] to archive 2023-08-07 08:55:08,266 DEBUG [StoreCloser-testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-07 08:55:08,267 DEBUG [StoreCloser-testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/fdc509227c7d46c8be6d0bcc598e74fe.ee46c734c4c3f049b054ec6f34f49d5c to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/fdc509227c7d46c8be6d0bcc598e74fe.ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:08,268 DEBUG [StoreCloser-testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/9468a53cdc51477a8f99ee801fbfbfbe.ee46c734c4c3f049b054ec6f34f49d5c to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/9468a53cdc51477a8f99ee801fbfbfbe.ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:08,269 DEBUG [StoreCloser-testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/a6d5906241f44f428b41b048d288a810.ee46c734c4c3f049b054ec6f34f49d5c to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/a6d5906241f44f428b41b048d288a810.ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:08,270 DEBUG [StoreCloser-testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/dcc4b5972743447992751542a226bcf9.ee46c734c4c3f049b054ec6f34f49d5c to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/dcc4b5972743447992751542a226bcf9.ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:08,271 DEBUG [StoreCloser-testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/06347138db5e40bfb279b6aca6133168.ee46c734c4c3f049b054ec6f34f49d5c to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/06347138db5e40bfb279b6aca6133168.ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:08,272 DEBUG [StoreCloser-testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/10b9da34b79345f28c8f6ae4106e9861.ee46c734c4c3f049b054ec6f34f49d5c to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/10b9da34b79345f28c8f6ae4106e9861.ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:08,273 DEBUG [StoreCloser-testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/8ed5067df3804d1c9e19e225c3c77c3f.ee46c734c4c3f049b054ec6f34f49d5c to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/8ed5067df3804d1c9e19e225c3c77c3f.ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:08,276 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-08-07 08:55:08,276 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-07 08:55:08,276 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c. 2023-08-07 08:55:08,276 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 7e7041dd5d1c44daa984633ad6291f9c: 2023-08-07 08:55:08,278 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 7e7041dd5d1c44daa984633ad6291f9c 2023-08-07 08:55:08,278 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=7e7041dd5d1c44daa984633ad6291f9c, regionState=CLOSED 2023-08-07 08:55:08,278 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691398508278"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691398508278"}]},"ts":"1691398508278"} 2023-08-07 08:55:08,280 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=154, resume processing ppid=153 2023-08-07 08:55:08,280 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure 7e7041dd5d1c44daa984633ad6291f9c, server=jenkins-hbase4.apache.org,34559,1691398396467 in 199 msec 2023-08-07 08:55:08,281 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=153, resume processing ppid=151 2023-08-07 08:55:08,281 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=7e7041dd5d1c44daa984633ad6291f9c, UNASSIGN in 202 msec 2023-08-07 08:55:08,282 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691398508282"}]},"ts":"1691398508282"} 2023-08-07 08:55:08,283 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-08-07 08:55:08,285 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-08-07 08:55:08,287 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 214 msec 2023-08-07 08:55:08,377 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=151 2023-08-07 08:55:08,377 INFO [Listener at localhost/33561] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-08-07 08:55:08,377 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegion 2023-08-07 08:55:08,378 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] procedure2.ProcedureExecutor(1029): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-08-07 08:55:08,380 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-08-07 08:55:08,380 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-08-07 08:55:08,380 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=156 2023-08-07 08:55:08,384 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:08,384 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c 2023-08-07 08:55:08,384 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71 2023-08-07 08:55:08,387 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/recovered.edits] 2023-08-07 08:55:08,387 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/recovered.edits] 2023-08-07 08:55:08,387 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2, FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/recovered.edits] 2023-08-07 08:55:08,390 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/7d3e573588254b6194cdeea22d25b24a to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf1/7d3e573588254b6194cdeea22d25b24a 2023-08-07 08:55:08,390 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/00697ed133ff4aeca8cd12bb2966fb51 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf1/00697ed133ff4aeca8cd12bb2966fb51 2023-08-07 08:55:08,391 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/273607c92aa046d892ba71005f5721fa to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/273607c92aa046d892ba71005f5721fa 2023-08-07 08:55:08,393 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/905852ab1029495fa27db79fa4c532e0 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/cf2/905852ab1029495fa27db79fa4c532e0 2023-08-07 08:55:08,393 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/43d72c1260c54bb8ab9ac76699e3a21a to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/43d72c1260c54bb8ab9ac76699e3a21a 2023-08-07 08:55:08,393 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/1031e67d4c424279bd02e6c8a096e288 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/cf2/1031e67d4c424279bd02e6c8a096e288 2023-08-07 08:55:08,394 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/9b60b41babdb445cb831ba1afe9cc3c5 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/9b60b41babdb445cb831ba1afe9cc3c5 2023-08-07 08:55:08,395 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/a72ba5bad0264a5183fbf79c7ae30801 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/a72ba5bad0264a5183fbf79c7ae30801 2023-08-07 08:55:08,396 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/recovered.edits/90.seqid to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71/recovered.edits/90.seqid 2023-08-07 08:55:08,396 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/recovered.edits/90.seqid to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c/recovered.edits/90.seqid 2023-08-07 08:55:08,396 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/cf538e5876ec330ccb93fc1a6d516e71 2023-08-07 08:55:08,396 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/7e7041dd5d1c44daa984633ad6291f9c 2023-08-07 08:55:08,397 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/c22b7ce84f684210984161264706990d to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/c22b7ce84f684210984161264706990d 2023-08-07 08:55:08,398 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/c68a9d6287694d9f8cb3baaa66d5c606 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/c68a9d6287694d9f8cb3baaa66d5c606 2023-08-07 08:55:08,399 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/cec48633c0224b26b022b9a3c532a69e to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf1/cec48633c0224b26b022b9a3c532a69e 2023-08-07 08:55:08,401 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/06347138db5e40bfb279b6aca6133168 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/06347138db5e40bfb279b6aca6133168 2023-08-07 08:55:08,402 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/10b9da34b79345f28c8f6ae4106e9861 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/10b9da34b79345f28c8f6ae4106e9861 2023-08-07 08:55:08,403 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/8ed5067df3804d1c9e19e225c3c77c3f to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/8ed5067df3804d1c9e19e225c3c77c3f 2023-08-07 08:55:08,404 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/9468a53cdc51477a8f99ee801fbfbfbe to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/9468a53cdc51477a8f99ee801fbfbfbe 2023-08-07 08:55:08,405 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/a6d5906241f44f428b41b048d288a810 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/a6d5906241f44f428b41b048d288a810 2023-08-07 08:55:08,406 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/dcc4b5972743447992751542a226bcf9 to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/dcc4b5972743447992751542a226bcf9 2023-08-07 08:55:08,406 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/fdc509227c7d46c8be6d0bcc598e74fe to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/cf2/fdc509227c7d46c8be6d0bcc598e74fe 2023-08-07 08:55:08,409 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/recovered.edits/85.seqid to hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/archive/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c/recovered.edits/85.seqid 2023-08-07 08:55:08,409 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/.tmp/data/default/testSplitTableRegion/ee46c734c4c3f049b054ec6f34f49d5c 2023-08-07 08:55:08,409 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-08-07 08:55:08,411 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-08-07 08:55:08,413 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-08-07 08:55:08,414 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-08-07 08:55:08,415 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-08-07 08:55:08,415 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-08-07 08:55:08,415 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691398508415"}]},"ts":"9223372036854775807"} 2023-08-07 08:55:08,415 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691398508415"}]},"ts":"9223372036854775807"} 2023-08-07 08:55:08,415 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691398508415"}]},"ts":"9223372036854775807"} 2023-08-07 08:55:08,417 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-08-07 08:55:08,417 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => ee46c734c4c3f049b054ec6f34f49d5c, NAME => 'testSplitTableRegion,,1691398504549.ee46c734c4c3f049b054ec6f34f49d5c.', STARTKEY => '', ENDKEY => ''}, {ENCODED => cf538e5876ec330ccb93fc1a6d516e71, NAME => 'testSplitTableRegion,,1691398506690.cf538e5876ec330ccb93fc1a6d516e71.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 7e7041dd5d1c44daa984633ad6291f9c, NAME => 'testSplitTableRegion,41,1691398506690.7e7041dd5d1c44daa984633ad6291f9c.', STARTKEY => '41', ENDKEY => ''}] 2023-08-07 08:55:08,417 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-08-07 08:55:08,417 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1691398508417"}]},"ts":"9223372036854775807"} 2023-08-07 08:55:08,418 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-08-07 08:55:08,419 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-08-07 08:55:08,420 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 42 msec 2023-08-07 08:55:08,482 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33357] master.MasterRpcServices(1230): Checking to see if procedure is done pid=156 2023-08-07 08:55:08,482 INFO [Listener at localhost/33561] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-08-07 08:55:08,496 INFO [Listener at localhost/33561] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=378 (was 377) - Thread LEAK? -, OpenFileDescriptor=829 (was 838), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=202 (was 202), ProcessCount=175 (was 175), AvailableMemoryMB=1739 (was 1805) 2023-08-07 08:55:08,496 INFO [Listener at localhost/33561] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-08-07 08:55:08,496 INFO [Listener at localhost/33561] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-08-07 08:55:08,497 DEBUG [Listener at localhost/33561] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x648b6fc0 to 127.0.0.1:57521 2023-08-07 08:55:08,497 DEBUG [Listener at localhost/33561] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 08:55:08,498 DEBUG [Listener at localhost/33561] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-08-07 08:55:08,498 DEBUG [Listener at localhost/33561] util.JVMClusterUtil(257): Found active master hash=1768762736, stopped=false 2023-08-07 08:55:08,498 INFO [Listener at localhost/33561] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,33357,1691398395153 2023-08-07 08:55:08,500 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-07 08:55:08,500 INFO [Listener at localhost/33561] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 08:55:08,500 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 08:55:08,500 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-07 08:55:08,500 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-07 08:55:08,500 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-07 08:55:08,501 DEBUG [Listener at localhost/33561] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x38c38623 to 127.0.0.1:57521 2023-08-07 08:55:08,501 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-07 08:55:08,501 DEBUG [Listener at localhost/33561] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 08:55:08,501 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-07 08:55:08,501 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-07 08:55:08,502 INFO [Listener at localhost/33561] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,46525,1691398396377' ***** 2023-08-07 08:55:08,502 INFO [Listener at localhost/33561] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-07 08:55:08,501 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-07 08:55:08,502 INFO [RS:0;jenkins-hbase4:46525] regionserver.HeapMemoryManager(220): Stopping 2023-08-07 08:55:08,502 INFO [RS:0;jenkins-hbase4:46525] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-07 08:55:08,502 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-07 08:55:08,502 INFO [RS:0;jenkins-hbase4:46525] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-07 08:55:08,503 INFO [RS:0;jenkins-hbase4:46525] regionserver.HRegionServer(3305): Received CLOSE for 261180939175fe040900ce2fa993a79f 2023-08-07 08:55:08,503 INFO [RS:0;jenkins-hbase4:46525] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:08,503 DEBUG [RS:0;jenkins-hbase4:46525] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x26051c8e to 127.0.0.1:57521 2023-08-07 08:55:08,503 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 261180939175fe040900ce2fa993a79f, disabling compactions & flushes 2023-08-07 08:55:08,503 DEBUG [RS:0;jenkins-hbase4:46525] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 08:55:08,503 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1691398398424.261180939175fe040900ce2fa993a79f. 2023-08-07 08:55:08,503 INFO [RS:0;jenkins-hbase4:46525] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-08-07 08:55:08,503 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1691398398424.261180939175fe040900ce2fa993a79f. 2023-08-07 08:55:08,503 DEBUG [RS:0;jenkins-hbase4:46525] regionserver.HRegionServer(1478): Online Regions={261180939175fe040900ce2fa993a79f=hbase:namespace,,1691398398424.261180939175fe040900ce2fa993a79f.} 2023-08-07 08:55:08,503 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1691398398424.261180939175fe040900ce2fa993a79f. after waiting 0 ms 2023-08-07 08:55:08,503 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1691398398424.261180939175fe040900ce2fa993a79f. 2023-08-07 08:55:08,503 INFO [Listener at localhost/33561] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,34633,1691398396427' ***** 2023-08-07 08:55:08,504 INFO [Listener at localhost/33561] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-07 08:55:08,504 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 261180939175fe040900ce2fa993a79f 1/1 column families, dataSize=78 B heapSize=488 B 2023-08-07 08:55:08,504 INFO [Listener at localhost/33561] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,34559,1691398396467' ***** 2023-08-07 08:55:08,504 INFO [RS:1;jenkins-hbase4:34633] regionserver.HeapMemoryManager(220): Stopping 2023-08-07 08:55:08,504 INFO [Listener at localhost/33561] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-07 08:55:08,504 INFO [RS:1;jenkins-hbase4:34633] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-07 08:55:08,505 INFO [RS:1;jenkins-hbase4:34633] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-07 08:55:08,504 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-07 08:55:08,504 INFO [RS:2;jenkins-hbase4:34559] regionserver.HeapMemoryManager(220): Stopping 2023-08-07 08:55:08,504 DEBUG [RS:0;jenkins-hbase4:46525] regionserver.HRegionServer(1504): Waiting on 261180939175fe040900ce2fa993a79f 2023-08-07 08:55:08,505 INFO [RS:2;jenkins-hbase4:34559] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-07 08:55:08,505 INFO [RS:2;jenkins-hbase4:34559] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-07 08:55:08,505 INFO [RS:2;jenkins-hbase4:34559] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:08,505 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-07 08:55:08,505 INFO [RS:1;jenkins-hbase4:34633] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:08,505 DEBUG [RS:1;jenkins-hbase4:34633] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x68f1cf0c to 127.0.0.1:57521 2023-08-07 08:55:08,505 DEBUG [RS:1;jenkins-hbase4:34633] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 08:55:08,505 DEBUG [RS:2;jenkins-hbase4:34559] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7649fe0f to 127.0.0.1:57521 2023-08-07 08:55:08,505 INFO [RS:1;jenkins-hbase4:34633] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,34633,1691398396427; all regions closed. 2023-08-07 08:55:08,505 DEBUG [RS:2;jenkins-hbase4:34559] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 08:55:08,506 INFO [RS:2;jenkins-hbase4:34559] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-07 08:55:08,506 INFO [RS:2;jenkins-hbase4:34559] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-07 08:55:08,506 INFO [RS:2;jenkins-hbase4:34559] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-07 08:55:08,506 INFO [RS:2;jenkins-hbase4:34559] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-08-07 08:55:08,507 INFO [RS:2;jenkins-hbase4:34559] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-08-07 08:55:08,507 DEBUG [RS:2;jenkins-hbase4:34559] regionserver.HRegionServer(1478): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-08-07 08:55:08,507 DEBUG [RS:2;jenkins-hbase4:34559] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-08-07 08:55:08,508 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-07 08:55:08,508 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-07 08:55:08,508 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-07 08:55:08,508 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-07 08:55:08,508 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-07 08:55:08,508 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=79.40 KB heapSize=125.77 KB 2023-08-07 08:55:08,520 DEBUG [RS:1;jenkins-hbase4:34633] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/oldWALs 2023-08-07 08:55:08,520 INFO [RS:1;jenkins-hbase4:34633] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34633%2C1691398396427:(num 1691398397961) 2023-08-07 08:55:08,520 DEBUG [RS:1;jenkins-hbase4:34633] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 08:55:08,520 INFO [RS:1;jenkins-hbase4:34633] regionserver.LeaseManager(133): Closed leases 2023-08-07 08:55:08,523 INFO [RS:1;jenkins-hbase4:34633] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-08-07 08:55:08,523 INFO [RS:1;jenkins-hbase4:34633] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-07 08:55:08,523 INFO [RS:1;jenkins-hbase4:34633] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-07 08:55:08,523 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-07 08:55:08,523 INFO [RS:1;jenkins-hbase4:34633] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-07 08:55:08,524 INFO [RS:1;jenkins-hbase4:34633] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:34633 2023-08-07 08:55:08,529 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/namespace/261180939175fe040900ce2fa993a79f/.tmp/info/d44742ac33a84773af668166f795da58 2023-08-07 08:55:08,532 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:08,532 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:08,532 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-07 08:55:08,533 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-07 08:55:08,533 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34633,1691398396427 2023-08-07 08:55:08,533 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-07 08:55:08,533 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-07 08:55:08,533 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/namespace/261180939175fe040900ce2fa993a79f/.tmp/info/d44742ac33a84773af668166f795da58 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/namespace/261180939175fe040900ce2fa993a79f/info/d44742ac33a84773af668166f795da58 2023-08-07 08:55:08,534 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.29 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/meta/1588230740/.tmp/info/1806ff0eb2cc41d7a21a837823c975d6 2023-08-07 08:55:08,535 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,34633,1691398396427] 2023-08-07 08:55:08,536 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,34633,1691398396427; numProcessing=1 2023-08-07 08:55:08,537 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,34633,1691398396427 already deleted, retry=false 2023-08-07 08:55:08,537 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,34633,1691398396427 expired; onlineServers=2 2023-08-07 08:55:08,538 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1806ff0eb2cc41d7a21a837823c975d6 2023-08-07 08:55:08,539 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/namespace/261180939175fe040900ce2fa993a79f/info/d44742ac33a84773af668166f795da58, entries=2, sequenceid=6, filesize=4.8 K 2023-08-07 08:55:08,540 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 261180939175fe040900ce2fa993a79f in 37ms, sequenceid=6, compaction requested=false 2023-08-07 08:55:08,541 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-07 08:55:08,541 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-07 08:55:08,541 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-07 08:55:08,544 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/namespace/261180939175fe040900ce2fa993a79f/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-08-07 08:55:08,544 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-07 08:55:08,545 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1691398398424.261180939175fe040900ce2fa993a79f. 2023-08-07 08:55:08,545 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 261180939175fe040900ce2fa993a79f: 2023-08-07 08:55:08,545 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1691398398424.261180939175fe040900ce2fa993a79f. 2023-08-07 08:55:08,548 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/meta/1588230740/.tmp/rep_barrier/22edb9ce153548bdb0761b2e13d2052d 2023-08-07 08:55:08,551 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 22edb9ce153548bdb0761b2e13d2052d 2023-08-07 08:55:08,561 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/meta/1588230740/.tmp/table/b945915d1bca4e9bbb4275be7d06437a 2023-08-07 08:55:08,564 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b945915d1bca4e9bbb4275be7d06437a 2023-08-07 08:55:08,564 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/meta/1588230740/.tmp/info/1806ff0eb2cc41d7a21a837823c975d6 as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/meta/1588230740/info/1806ff0eb2cc41d7a21a837823c975d6 2023-08-07 08:55:08,568 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1806ff0eb2cc41d7a21a837823c975d6 2023-08-07 08:55:08,568 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/meta/1588230740/info/1806ff0eb2cc41d7a21a837823c975d6, entries=59, sequenceid=210, filesize=11.1 K 2023-08-07 08:55:08,568 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/meta/1588230740/.tmp/rep_barrier/22edb9ce153548bdb0761b2e13d2052d as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/meta/1588230740/rep_barrier/22edb9ce153548bdb0761b2e13d2052d 2023-08-07 08:55:08,571 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 22edb9ce153548bdb0761b2e13d2052d 2023-08-07 08:55:08,572 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/meta/1588230740/rep_barrier/22edb9ce153548bdb0761b2e13d2052d, entries=24, sequenceid=210, filesize=7.5 K 2023-08-07 08:55:08,572 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/meta/1588230740/.tmp/table/b945915d1bca4e9bbb4275be7d06437a as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/meta/1588230740/table/b945915d1bca4e9bbb4275be7d06437a 2023-08-07 08:55:08,575 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b945915d1bca4e9bbb4275be7d06437a 2023-08-07 08:55:08,575 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/meta/1588230740/table/b945915d1bca4e9bbb4275be7d06437a, entries=36, sequenceid=210, filesize=8.0 K 2023-08-07 08:55:08,576 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 68ms, sequenceid=210, compaction requested=false 2023-08-07 08:55:08,580 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-08-07 08:55:08,581 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-07 08:55:08,581 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-07 08:55:08,582 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-07 08:55:08,582 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-07 08:55:08,582 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-08-07 08:55:08,636 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 08:55:08,636 INFO [RS:1;jenkins-hbase4:34633] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,34633,1691398396427; zookeeper connection closed. 2023-08-07 08:55:08,636 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34633-0x101dea1d48f0002, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 08:55:08,637 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@7bbe1a62] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@7bbe1a62 2023-08-07 08:55:08,705 INFO [RS:0;jenkins-hbase4:46525] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,46525,1691398396377; all regions closed. 2023-08-07 08:55:08,707 INFO [RS:2;jenkins-hbase4:34559] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,34559,1691398396467; all regions closed. 2023-08-07 08:55:08,716 DEBUG [RS:0;jenkins-hbase4:46525] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/oldWALs 2023-08-07 08:55:08,716 INFO [RS:0;jenkins-hbase4:46525] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C46525%2C1691398396377:(num 1691398397960) 2023-08-07 08:55:08,716 DEBUG [RS:2;jenkins-hbase4:34559] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/oldWALs 2023-08-07 08:55:08,716 DEBUG [RS:0;jenkins-hbase4:46525] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 08:55:08,716 INFO [RS:2;jenkins-hbase4:34559] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34559%2C1691398396467.meta:.meta(num 1691398398139) 2023-08-07 08:55:08,716 INFO [RS:0;jenkins-hbase4:46525] regionserver.LeaseManager(133): Closed leases 2023-08-07 08:55:08,716 INFO [RS:0;jenkins-hbase4:46525] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-08-07 08:55:08,717 INFO [RS:0;jenkins-hbase4:46525] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-07 08:55:08,717 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-07 08:55:08,717 INFO [RS:0;jenkins-hbase4:46525] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-07 08:55:08,717 INFO [RS:0;jenkins-hbase4:46525] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-07 08:55:08,718 INFO [RS:0;jenkins-hbase4:46525] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:46525 2023-08-07 08:55:08,720 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:08,720 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-07 08:55:08,720 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,46525,1691398396377 2023-08-07 08:55:08,721 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/WALs/jenkins-hbase4.apache.org,34559,1691398396467/jenkins-hbase4.apache.org%2C34559%2C1691398396467.1691398397961 not finished, retry = 0 2023-08-07 08:55:08,723 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,46525,1691398396377] 2023-08-07 08:55:08,723 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,46525,1691398396377; numProcessing=2 2023-08-07 08:55:08,730 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,46525,1691398396377 already deleted, retry=false 2023-08-07 08:55:08,730 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,46525,1691398396377 expired; onlineServers=1 2023-08-07 08:55:08,813 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-08-07 08:55:08,813 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-08-07 08:55:08,822 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 08:55:08,822 INFO [RS:0;jenkins-hbase4:46525] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,46525,1691398396377; zookeeper connection closed. 2023-08-07 08:55:08,822 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:46525-0x101dea1d48f0001, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 08:55:08,823 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@3804a5d5] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@3804a5d5 2023-08-07 08:55:08,824 DEBUG [RS:2;jenkins-hbase4:34559] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/oldWALs 2023-08-07 08:55:08,824 INFO [RS:2;jenkins-hbase4:34559] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34559%2C1691398396467:(num 1691398397961) 2023-08-07 08:55:08,824 DEBUG [RS:2;jenkins-hbase4:34559] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 08:55:08,824 INFO [RS:2;jenkins-hbase4:34559] regionserver.LeaseManager(133): Closed leases 2023-08-07 08:55:08,824 INFO [RS:2;jenkins-hbase4:34559] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-08-07 08:55:08,824 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-07 08:55:08,825 INFO [RS:2;jenkins-hbase4:34559] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:34559 2023-08-07 08:55:08,827 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34559,1691398396467 2023-08-07 08:55:08,827 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-07 08:55:08,828 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,34559,1691398396467] 2023-08-07 08:55:08,828 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,34559,1691398396467; numProcessing=3 2023-08-07 08:55:08,928 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 08:55:08,928 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): regionserver:34559-0x101dea1d48f0003, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 08:55:08,928 INFO [RS:2;jenkins-hbase4:34559] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,34559,1691398396467; zookeeper connection closed. 2023-08-07 08:55:08,929 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@16dc49dc] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@16dc49dc 2023-08-07 08:55:08,929 INFO [Listener at localhost/33561] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-08-07 08:55:08,930 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,34559,1691398396467 already deleted, retry=false 2023-08-07 08:55:08,930 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,34559,1691398396467 expired; onlineServers=0 2023-08-07 08:55:08,930 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,33357,1691398395153' ***** 2023-08-07 08:55:08,930 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-08-07 08:55:08,931 DEBUG [M:0;jenkins-hbase4:33357] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2850274b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-07 08:55:08,931 INFO [M:0;jenkins-hbase4:33357] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,33357,1691398395153 2023-08-07 08:55:08,931 INFO [M:0;jenkins-hbase4:33357] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,33357,1691398395153; all regions closed. 2023-08-07 08:55:08,931 DEBUG [M:0;jenkins-hbase4:33357] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 08:55:08,931 DEBUG [M:0;jenkins-hbase4:33357] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-08-07 08:55:08,931 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-08-07 08:55:08,931 DEBUG [M:0;jenkins-hbase4:33357] cleaner.HFileCleaner(317): Stopping file delete threads 2023-08-07 08:55:08,931 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691398397590] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691398397590,5,FailOnTimeoutGroup] 2023-08-07 08:55:08,931 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691398397590] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691398397590,5,FailOnTimeoutGroup] 2023-08-07 08:55:08,932 INFO [M:0;jenkins-hbase4:33357] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-08-07 08:55:08,932 INFO [M:0;jenkins-hbase4:33357] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-08-07 08:55:08,932 INFO [M:0;jenkins-hbase4:33357] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-08-07 08:55:08,932 DEBUG [M:0;jenkins-hbase4:33357] master.HMaster(1512): Stopping service threads 2023-08-07 08:55:08,932 INFO [M:0;jenkins-hbase4:33357] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-08-07 08:55:08,932 ERROR [M:0;jenkins-hbase4:33357] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] Thread[HFileArchiver-14,5,PEWorkerGroup] Thread[HFileArchiver-15,5,PEWorkerGroup] Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-08-07 08:55:08,932 INFO [M:0;jenkins-hbase4:33357] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 08:55:08,933 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-08-07 08:55:08,933 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-08-07 08:55:08,933 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-07 08:55:08,933 DEBUG [M:0;jenkins-hbase4:33357] zookeeper.ZKUtil(398): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-08-07 08:55:08,933 WARN [M:0;jenkins-hbase4:33357] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-08-07 08:55:08,933 INFO [M:0;jenkins-hbase4:33357] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 08:55:08,934 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-07 08:55:08,934 INFO [M:0;jenkins-hbase4:33357] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-08-07 08:55:08,934 DEBUG [M:0;jenkins-hbase4:33357] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-07 08:55:08,934 INFO [M:0;jenkins-hbase4:33357] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 08:55:08,934 DEBUG [M:0;jenkins-hbase4:33357] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 08:55:08,934 DEBUG [M:0;jenkins-hbase4:33357] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-07 08:55:08,934 DEBUG [M:0;jenkins-hbase4:33357] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 08:55:08,934 INFO [M:0;jenkins-hbase4:33357] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=608.17 KB heapSize=719.06 KB 2023-08-07 08:55:08,939 ERROR [BP-1940969354-172.31.14.131-1691398391279 heartbeating to localhost/127.0.0.1:45787] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1940969354-172.31.14.131-1691398391279 (Datanode Uuid 5daa8478-5963-4c33-b92f-e21f18db34d9) service to localhost/127.0.0.1:45787 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-08-07 08:55:08,955 INFO [M:0;jenkins-hbase4:33357] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=608.17 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/d17056ddc39b4bb0965470343c31fb9d 2023-08-07 08:55:08,959 INFO [M:0;jenkins-hbase4:33357] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d17056ddc39b4bb0965470343c31fb9d 2023-08-07 08:55:08,960 DEBUG [M:0;jenkins-hbase4:33357] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/d17056ddc39b4bb0965470343c31fb9d as hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/d17056ddc39b4bb0965470343c31fb9d 2023-08-07 08:55:08,963 INFO [M:0;jenkins-hbase4:33357] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d17056ddc39b4bb0965470343c31fb9d 2023-08-07 08:55:08,963 INFO [M:0;jenkins-hbase4:33357] regionserver.HStore(1080): Added hdfs://localhost:45787/user/jenkins/test-data/3eda61e0-5182-0d9e-4a9d-5dfb7f284b04/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/d17056ddc39b4bb0965470343c31fb9d, entries=156, sequenceid=1282, filesize=34.9 K 2023-08-07 08:55:08,964 INFO [M:0;jenkins-hbase4:33357] regionserver.HRegion(2948): Finished flush of dataSize ~608.17 KB/622765, heapSize ~719.05 KB/736304, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 30ms, sequenceid=1282, compaction requested=false 2023-08-07 08:55:08,964 INFO [M:0;jenkins-hbase4:33357] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 08:55:08,964 DEBUG [M:0;jenkins-hbase4:33357] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-07 08:55:08,968 INFO [M:0;jenkins-hbase4:33357] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-08-07 08:55:08,968 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-07 08:55:08,969 INFO [M:0;jenkins-hbase4:33357] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33357 2023-08-07 08:55:08,970 DEBUG [M:0;jenkins-hbase4:33357] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,33357,1691398395153 already deleted, retry=false 2023-08-07 08:55:09,072 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 08:55:09,072 INFO [M:0;jenkins-hbase4:33357] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,33357,1691398395153; zookeeper connection closed. 2023-08-07 08:55:09,072 DEBUG [Listener at localhost/33561-EventThread] zookeeper.ZKWatcher(600): master:33357-0x101dea1d48f0000, quorum=127.0.0.1:57521, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 08:55:09,074 WARN [Listener at localhost/33561] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-07 08:55:09,079 INFO [Listener at localhost/33561] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-07 08:55:09,186 WARN [BP-1940969354-172.31.14.131-1691398391279 heartbeating to localhost/127.0.0.1:45787] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1940969354-172.31.14.131-1691398391279 (Datanode Uuid 5daa8478-5963-4c33-b92f-e21f18db34d9) service to localhost/127.0.0.1:45787 2023-08-07 08:55:09,190 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/cluster_383c75a2-fb83-c721-ca60-8c2ce8eb403c/dfs/data/data5/current/BP-1940969354-172.31.14.131-1691398391279] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 08:55:09,190 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/cluster_383c75a2-fb83-c721-ca60-8c2ce8eb403c/dfs/data/data6/current/BP-1940969354-172.31.14.131-1691398391279] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 08:55:09,192 WARN [Listener at localhost/33561] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-07 08:55:09,196 INFO [Listener at localhost/33561] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-07 08:55:09,302 WARN [BP-1940969354-172.31.14.131-1691398391279 heartbeating to localhost/127.0.0.1:45787] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-07 08:55:09,302 WARN [BP-1940969354-172.31.14.131-1691398391279 heartbeating to localhost/127.0.0.1:45787] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1940969354-172.31.14.131-1691398391279 (Datanode Uuid 0712b291-8bcc-4f35-bb85-15469634c482) service to localhost/127.0.0.1:45787 2023-08-07 08:55:09,303 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/cluster_383c75a2-fb83-c721-ca60-8c2ce8eb403c/dfs/data/data3/current/BP-1940969354-172.31.14.131-1691398391279] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 08:55:09,303 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/cluster_383c75a2-fb83-c721-ca60-8c2ce8eb403c/dfs/data/data4/current/BP-1940969354-172.31.14.131-1691398391279] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 08:55:09,305 WARN [Listener at localhost/33561] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-07 08:55:09,308 INFO [Listener at localhost/33561] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-07 08:55:09,411 WARN [BP-1940969354-172.31.14.131-1691398391279 heartbeating to localhost/127.0.0.1:45787] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-07 08:55:09,411 WARN [BP-1940969354-172.31.14.131-1691398391279 heartbeating to localhost/127.0.0.1:45787] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1940969354-172.31.14.131-1691398391279 (Datanode Uuid 4fd49325-1f52-494b-bfd8-92875de86a16) service to localhost/127.0.0.1:45787 2023-08-07 08:55:09,412 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/cluster_383c75a2-fb83-c721-ca60-8c2ce8eb403c/dfs/data/data1/current/BP-1940969354-172.31.14.131-1691398391279] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 08:55:09,412 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d44aef05-e41a-7731-4ff1-a60d77859388/cluster_383c75a2-fb83-c721-ca60-8c2ce8eb403c/dfs/data/data2/current/BP-1940969354-172.31.14.131-1691398391279] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 08:55:09,435 INFO [Listener at localhost/33561] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-07 08:55:09,624 INFO [Listener at localhost/33561] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-08-07 08:55:09,861 INFO [Listener at localhost/33561] hbase.HBaseTestingUtility(1293): Minicluster is down